-- Logs begin at Mon 2022-06-27 14:41:22 UTC, end at Mon 2022-06-27 15:47:32 UTC. -- Jun 27 15:11:35.389410 dv-cinder-847798-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 15:11:35.402201 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 15:11:35 2022] *** Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: nodename: dv-cinder-847798-1-iscsi Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: machine: x86_64 Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: clock source: unix Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: pcre jit disabled Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: detected number of CPU cores: 4 Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: current working directory: / Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: detected binary path: /usr/bin/uwsgi-core Jun 27 15:11:35.405296 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: your processes number limit is 47680 Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: your memory page size is 4096 bytes Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: detected max file descriptor number: 2048 Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: lock engine: pthread robust mutexes Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: thunder lock: enabled Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 15:11:35.406389 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 15:11:35.459187 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: Python main interpreter initialized at 0x55fdc6d169c0 Jun 27 15:11:35.459187 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: python threads support enabled Jun 27 15:11:35.459187 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: your server socket listen backlog is limited to 100 connections Jun 27 15:11:35.459187 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: your mercy for graceful operations on workers is 90 seconds Jun 27 15:11:35.459700 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 15:11:35.459809 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: *** Operational MODE: preforking *** Jun 27 15:11:35.460124 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: *** uWSGI is running in multiple interpreter mode *** Jun 27 15:11:35.460124 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: spawned uWSGI master process (pid: 80200) Jun 27 15:11:35.460418 dv-cinder-847798-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 15:11:35.466403 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: spawned uWSGI worker 1 (pid: 80212, cores: 1) Jun 27 15:11:35.466403 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: spawned uWSGI worker 2 (pid: 80213, cores: 1) Jun 27 15:11:35.466403 dv-cinder-847798-1-iscsi devstack@keystone.service[80200]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 15:11:43.267175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:11:43.267175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Jun 27 15:11:43.268023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:11:43.268023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Jun 27 15:11:43.342126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:11:43.342126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: from cryptography.utils import int_from_bytes Jun 27 15:11:43.342789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:11:43.342789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: from cryptography.utils import int_from_bytes Jun 27 15:11:43.351947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 15:11:43.352250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 15:11:43.352584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] command line args: [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 15:11:43.353020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 15:11:43.353302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ================================================================================ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 15:11:43.353719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.354126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_dir = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.354433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.354739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_source = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.355030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] debug = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.355354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.356610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.356610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.356610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.357068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.357451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.357732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_config_append = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.358029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.358331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_dir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.358626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.359048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.359657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.360127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.360615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.360861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.361246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.361662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.362077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.362485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.362882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.363284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.363684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.364106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.364561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.365036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.365875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.366183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] publish_errors = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.366580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.366978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.367288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.367711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.368116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.368543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] standard_threads = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.368947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.369370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.369663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.369970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_journal = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.370275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_json = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.370580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_stderr = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.370885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_syslog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.371285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.371728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.372140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.372621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.373036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.373469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.373876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.374303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.374754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.375232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.375661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.376108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.376547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.376978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.377376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.377803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.378231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.378644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.379087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.379528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.379951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.380384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.380829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.381275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.381716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.382154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.382553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.382952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.383346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.383763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.384193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.384648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.385046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.385444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.385871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.386298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.386721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.387143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.387548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.387947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.388377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.388777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.389178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.389584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.389980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.390528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.390997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.391435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.391846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.392254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.392688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.393097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.393512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.393909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.394325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.394723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.395123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.395531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.395933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.396341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.396779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.397194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.397598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.398002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.398449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.398848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.399241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.399652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.400051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.400534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.400948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.401369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.401780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.402198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.402595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.402996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.403414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.403818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.404212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.404660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.405067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.405464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.405863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.406307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.406758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.407234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.407641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.408042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.408469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.408868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.409265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.409652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.410043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.410429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.410808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.411201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.411601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.411982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.412414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.412807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.413227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.413619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.414044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.414484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.414887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.415275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.external = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.415653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.416161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.416589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.416987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.password = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.417379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.token = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.417785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.418182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.418593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.418980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.419370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.419759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.420137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.420557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.420965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.421357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.421921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.422356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.422792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.423166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.423579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.423978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.424379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.424787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.425230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.425615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.426019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.426394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.426796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.427165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.427535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.427921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.428345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.428732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.428973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 15:11:43.429148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.429269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 15:11:43.429549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] command line args: [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 15:11:43.429626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.429955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 15:11:43.430071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.430234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ================================================================================ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 15:11:43.430359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.430639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] admin_token = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.430778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.431030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_dir = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.431177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.431332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.432366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.432532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_source = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.432532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] debug = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.432726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.432821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.432977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.433094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.433211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.433323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.433440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.433559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.433679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.433799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.433965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.434093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.434257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_config_append = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.434366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.434552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.434729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.434846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_dir = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.435121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.435238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.435504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.435612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.435875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.435992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.436245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.436382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.436648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.436766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.436940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.437058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.437284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.437878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.438325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.438446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.438766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.438884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.439185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.439301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.439567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.439689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.439953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.440068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.440373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.440494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.440733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.440862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.441112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.441227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.441484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.441600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.441862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.442262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.442379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.442657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] publish_errors = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.442771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.443031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.443153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.443392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.443501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.443686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.443799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.444091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.444207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.444508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.444977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.445249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] standard_threads = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.445363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.445636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.445754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.446039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.446155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.446336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.446459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.446655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_journal = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.446825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.446956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_json = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.447184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_stderr = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.447304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.447481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_syslog = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.447607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.447862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:11:43.447987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.448288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.448419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.448708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.448822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.449125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.449252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.449509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.449625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.449891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.450005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.450266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.450390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.450668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.450787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.451941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.452201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.452337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.452632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.452750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.453102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.453248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.453560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.453977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.454367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.454492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.454769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.454888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.455171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.455295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.455553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.455676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.455935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.456059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.456356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.456477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.456730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.456847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.457110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.457236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.457490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.457608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.457899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.458016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.458300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.458416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.458702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.458824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.459952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.460220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.460356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.460635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.460753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.461881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.462154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.462276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.462556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.462682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.462940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.463057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.463343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.463466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.463716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.463837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.464988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.465247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.465367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.465622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.465739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.465999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.466115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.466373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.466491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.466753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.466870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.467126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.467250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.467517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.467633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.467900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.468016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.468324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.468484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.468740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.469278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.469450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.469650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.469872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.470086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.470417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.470633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.470846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.470846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.471247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.471459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.471459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.471889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.472105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.472105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.472637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.472875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.472875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.473275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.473483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.473483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.473880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.474106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.474317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.474542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.474542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.474941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.475151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.475151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.475564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.475785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.475785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.476184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.476433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.476668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.476884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.476884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.477313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.477538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.477538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.477949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.478161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.478161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.478565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.478772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.478772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.479194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.479405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.479405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.479802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.480011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.480011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.480600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.480812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.480812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.481221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.481434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.481434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.481849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.482061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.482061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.482458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.482665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.482665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.483911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.484551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.484762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.484762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.485181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.485397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.485397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.485809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.486020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.486020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.486431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.486643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.486643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.487050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.487260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.487260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.487653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.487864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.488989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.489399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.489609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.489609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.490020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.490233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.490445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.490659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.490659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.491069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.491304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.491304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.491845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.492066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.492066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.492525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.492738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.492949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.493158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.493158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.493553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.493779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.493779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.494178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.494399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.494608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.494818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.494818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.495247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.external = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.password = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.497022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.token = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.498763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.499975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.499975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.499975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.500608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.500927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.501312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.501696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.502065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.502468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.502861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.503235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.503634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.504052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.504482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.504884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.505261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.505665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.506042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.506412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.506800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.507188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.507554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.507939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.508334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.508737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.509109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.509500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.509876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.510308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.510685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.511072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.511457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.511829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.512201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.512644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.513022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.513395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.513782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.514168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.514532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.514907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.515300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.515705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.516098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.516592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.517019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.517482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.517852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.518227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.518599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.518961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.519330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.519700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.520078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.520301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.520301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.520770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.520987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.520987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.521446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.521819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.522203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.522587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.522966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.523352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.523733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.524098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.524519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.524886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.525273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.525645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.526029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.526401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.526773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.527161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.527552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.527916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.528278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.528690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.529074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.529348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.529594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.529909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.530276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.530640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.531070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.531469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.531839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.532222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.532643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.533031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.533398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.533760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.534138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.534505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.534875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.535244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.535628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.536010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.536435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.536818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.537199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.537570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.537961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.538332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.538711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.539084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.539453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.539842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.540226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.540641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.541041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.541405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.541784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.542156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.542521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.542894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.543285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.543660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.544026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.544444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.544821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.545215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.545607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.545974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.546364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.546862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.547233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.547616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.547992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.548396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.548765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.549149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.549514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.549902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.550294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.550755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.551110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.551499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.551882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.552264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.552698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.553062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.553446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.553830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.554214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.554606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.555166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.555528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.555911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.556307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.556713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.557102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.557467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.557854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.558236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.558607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.558832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.559068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.559285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.559501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.559829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.560216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.560652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.561056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.561428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.561799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.562205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.562574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.562947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.563334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.563703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.564093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.564509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.564878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.565256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.565658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.566043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.566410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.566778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.567141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.567519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.567904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.568309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.568719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.569176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.569933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.570318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.570704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.571080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.571457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.571820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.572186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.572587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.572957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.573344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.573713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.574090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.574463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.574843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.575208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.575634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.575997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.576410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.576784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:11:43.577077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 15:11:43.606367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.606700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.607016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.617228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.638788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.639098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.716257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.772255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.772611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.776375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.813280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.813564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.841355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.841608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.841788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.842005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.842221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.842481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.857323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.857575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.857775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.857962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.858178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.858394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.869308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.869533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.869762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.869962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.870174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.870388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.870596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.874093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.874325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.874545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.874843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.875063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.875283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.875496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.882922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.883601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.896450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.896724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.896868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.897882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.898007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.898129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:11:43.916040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.916281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:11:43.916590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.916787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:11:43.916967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.917164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.917412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:11:43.917539 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.917669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.917788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.917930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:11:43.918060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.918190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.918190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.918321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.918573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.929191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.929475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.929726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.929971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.930060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.930265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.930535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.930808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.938663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.939097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.941968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.942453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.942575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.942702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.942788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.942930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.943866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.944081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.944305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.944433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.944530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.944977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.945040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.945100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.945283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.945547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.972295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.972554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.973940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.974096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.974238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.974397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.979440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.979601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.979836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.979931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.980114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.980287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.987631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.987801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.987934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.988094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.992492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.992758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.992925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:43.993043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.993992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.994106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.994283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.994405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.994601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:43.994926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.013816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.014130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.016922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.017214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.019197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.021841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.022912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.023398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.025520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.025818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.028288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.030922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.032285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.034744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.036074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.036386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.036671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.036953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.037233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.037489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.038456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.038764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.039046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.039344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.039626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.039901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.056989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.057284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.057557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.057814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.059150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.059452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.059722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.060003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.066461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.068126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.068389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.068492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.070278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.070556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.072071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.072313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.072641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.072917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.073163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.073721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.074002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.074288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.074569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.074830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.086944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.087227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.087545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.087819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.089624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.089914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.090204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.090999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.091296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.091712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.092176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.092647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.093102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.093636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.108549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.112419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.113968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.115736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.117316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.117505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.119133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.119406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.119680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.120693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.122325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.123978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.124257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.124577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.127878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.129622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.132276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.132995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.133353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.133814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.133949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.134198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.137271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.137561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.137846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.138118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.144506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.144875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.145216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.147548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.147846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.148128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.153407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.153679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.153952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.154237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.154498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.154881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.155167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.155441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.155710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:11:44.156445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.156997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.157121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.157251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.157388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.157503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.157786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.158077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.158348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.158619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.158887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.159154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.159541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.159806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:11:44.196711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x55fdc6d169c0 pid: 80213 (default app) Jun 27 15:11:44.198474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x55fdc6d169c0 pid: 80212 (default app) Jun 27 15:11:44.200639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cdf67-7745-490c-ba68-fad3a152ea31 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:44.200942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cdf67-7745-490c-ba68-fad3a152ea31 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:44.201220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cdf67-7745-490c-ba68-fad3a152ea31 None None] PATH_INFO: `/v3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:44.206702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Jun 27 15:11:44 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:11:52.373861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:52.375374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:52.375374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:52.379692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:11:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:11:52.386112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:52.386595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:52.386756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:52.461438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: INFO dbcounter [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] Registered counter for database keystone Jun 27 15:11:52.485257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG oslo_db.sqlalchemy.engines [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80213) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 15:11:52.493061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writer thread running {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 15:11:52.689226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80213) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 15:11:52.689625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80213) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 15:11:52.747075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:11:52.748758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:11:52.904122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:11:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 15:11:53.026530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:53.098192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: INFO dbcounter [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] Registered counter for database keystone Jun 27 15:11:53.119641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG oslo_db.sqlalchemy.engines [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80212) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 15:11:53.128029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writer thread running {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 15:11:53.253177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5c177-2326-417a-80a4-175a2200cb46 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:53.257862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:53.258597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:53.258925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:53.259246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:53.269375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:53.323631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 15:11:53.323850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30dd50-a3d3-4a69-95e5-464d325c0cf9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:53.335666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jun 27 15:11:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:11:53.342008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:53.358151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0383-91b0-4b8c-969d-7cf7ea9d1272 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:53.363974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:53.365324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:53.365663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:53.366000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] PATH_INFO: `/v3/projects/9702a09cdfa2459d8590360a884e14e2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:53.371115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] RBAC: Authorizing `identity:get_project(project_id=9702a09cdfa2459d8590360a884e14e2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:53.422387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 15:11:53.423081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4db3dd-3125-4ba8-b0a2-b79badf23866 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:53.427576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Jun 27 15:11:53 2022] GET /identity/v3/projects/9702a09cdfa2459d8590360a884e14e2 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:11:55.541331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025300d5-e0a6-4096-a27c-62bf039858af None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:55.541920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025300d5-e0a6-4096-a27c-62bf039858af None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:55.541920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025300d5-e0a6-4096-a27c-62bf039858af None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:55.546169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:11:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:11:55.552791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:55.553322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:55.553432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:55.574817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:11:55.576502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:11:55.585688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:11:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 15:11:55.608620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025300d5-e0a6-4096-a27c-62bf039858af None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:55.623147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025300d5-e0a6-4096-a27c-62bf039858af None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:55.628902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:55.629566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:55.629847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:55.630120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:55.639085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:55.641702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63558374-c8f4-41ce-8c7a-9afd34ce7308 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:55.651585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jun 27 15:11:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:11:55.658655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:55.668858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995e73-1fc7-46cb-aa44-c3d1532433d6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:55.674665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:55.675629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:55.675979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:55.676303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:55.678034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:55.679966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f07f-4e0d-4c1a-916e-d8eeb434c0df admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:55.681636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Jun 27 15:11:55 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:11:57.844110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aae87-2739-43c4-8e42-88a39d0057fb None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:57.844968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aae87-2739-43c4-8e42-88a39d0057fb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:57.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aae87-2739-43c4-8e42-88a39d0057fb None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:57.847344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:11:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:11:57.857074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:57.857873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:57.857873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:57.876850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:11:57.878178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:11:57.922920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93c14b-a2f0-4a70-ba2c-6cd43a0e484e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:57.923519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93c14b-a2f0-4a70-ba2c-6cd43a0e484e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:57.924013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93c14b-a2f0-4a70-ba2c-6cd43a0e484e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:57.926829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:11:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:11:57.934155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:57.934665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:57.935174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:57.972618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:11:57.996779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.011451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG passlib.handlers.bcrypt [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80212) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 15:11:58.012142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG passlib.handlers.bcrypt [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80212) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 15:11:58.024244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.065475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:11:58.068417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:11:58.122346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350473c6-12df-4984-866b-c9b71adb03f5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.173426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.174269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.174556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.174842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.191380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:11:58.204288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:11:58.206259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:11:58.207676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 7/13] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:11:58.211774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.236135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 7/14] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:11:58.260811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.332941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.366957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d71e8-0f8e-4806-8943-58957a525f9b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.387944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6752e4-5dbd-431e-a18d-79166116ae8f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.390632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.424300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.425014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.425295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.425575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.426363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:11:58.426669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:58.428145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:58.439120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.440558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.441085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.441582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.441786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:11:58.448106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.546646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:58.548202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:58.580804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c3a37-1955-4582-9fc2-b95d9a219874 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.631349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.632955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.633462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.633963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.636454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:58.638948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:58.642834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 9/16] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:11:58.649277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.663437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 8/17] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:11:58.669687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.673383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.676795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.784442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aab01f-0e89-4eb4-bfe9-e89b39198b3c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.798165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f87e2-56af-4a2b-a44e-b2b8f7365a9c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:58.822220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.823038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.823329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.823589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.845004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:58.846523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:58.847156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:58.847984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:58.848268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:58.848787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:58.850358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:58.851842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:58.853740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:11:58.862145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:11:58.875889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:11:58.876865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:11:58.877788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:11:58.877788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:11:58.877788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:11:58.877788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:11:58.877788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:11:58.883258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:11:58.944026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.966689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:11:58.989595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adac246-d34a-44fc-ba75-4ccca2bbe908 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:59.002553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954355ec-8822-4d8a-9990-483ed446980e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:11:59.026645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:59.027313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:59.027599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:59.027876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:59.028765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.domain.id=default, user.id=d7a4d53e682240199113e94049a9ab62) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:11:59.029106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:59.030588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bd1ac-1235-45a1-812c-c1fe389c1fb1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:59.038306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/role_assignments?user.id=d7a4d53e682240199113e94049a9ab62&scope.domain.id=default&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 233 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:11:59.044817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:11:59.045537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:11:59.045823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:11:59.046105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:11:59.046994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:11:59.047311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:11:59.048892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e616e-ffc5-4e2d-8b30-0ca495964285 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:11:59.096362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:02.067703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5467-01fc-46f4-9453-bb2a4aaa58c2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.068992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5467-01fc-46f4-9453-bb2a4aaa58c2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.069837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5467-01fc-46f4-9453-bb2a4aaa58c2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.071490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:02.088305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.088836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.088905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.113413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:02.114761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:02.201267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:02.270528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d5467-01fc-46f4-9453-bb2a4aaa58c2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:02.410192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d5467-01fc-46f4-9453-bb2a4aaa58c2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:02.460172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:02.461406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.461888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.462355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.478123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-08cffc05-e3ab-40a0-8c1e-3ff767dd234d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:02.489824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:02.490753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:02.491498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:02.494518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 13/24] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:02.505537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:02.580674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:02.602654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f9758-0be1-4b7f-89c2-f8e79405a647 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.603278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f9758-0be1-4b7f-89c2-f8e79405a647 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.603752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f9758-0be1-4b7f-89c2-f8e79405a647 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.606010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 14/25] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:02.614645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e556-bcf4-432d-9ee3-6b358c28a6a2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.615253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e556-bcf4-432d-9ee3-6b358c28a6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.615723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e556-bcf4-432d-9ee3-6b358c28a6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.628715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:02.656405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-92f0e556-bcf4-432d-9ee3-6b358c28a6a2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:02.658008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-92f0e556-bcf4-432d-9ee3-6b358c28a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:02.721853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264950c4-4e5a-4e82-b420-537a6f03fdff None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:02.801784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:02.802872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.803424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.803907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.804915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:02.805427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:02.807157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459fc9b-5b7c-4bb0-a97c-25cba4646ceb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:02.823290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 15/26] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:02.830834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc5a-a952-4f80-bf8d-f072a95ac1f9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.831338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc5a-a952-4f80-bf8d-f072a95ac1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.831824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240dc5a-a952-4f80-bf8d-f072a95ac1f9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.834752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 16/27] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:02.837183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 12/28] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:02.844180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240dc5a-a952-4f80-bf8d-f072a95ac1f9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:02.848967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ff65-8ff6-4811-97c7-10d8f8eb8d86 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.849267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ff65-8ff6-4811-97c7-10d8f8eb8d86 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.849553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ff65-8ff6-4811-97c7-10d8f8eb8d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.904256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-79f0ff65-8ff6-4811-97c7-10d8f8eb8d86 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:02.905556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-79f0ff65-8ff6-4811-97c7-10d8f8eb8d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:02.984835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 13/29] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:02.987209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f5bb7-4cd7-4196-b33e-d40622fd3a43 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:02.987760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f5bb7-4cd7-4196-b33e-d40622fd3a43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:02.988245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f5bb7-4cd7-4196-b33e-d40622fd3a43 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:02.990445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 14/30] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:02.991211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f5bb7-4cd7-4196-b33e-d40622fd3a43 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.013091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240dc5a-a952-4f80-bf8d-f072a95ac1f9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.046863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.067784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.070626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.071144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.071764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.073559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.076193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bb7b8-d545-4349-aca4-bce5453ac6f1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.078268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 17/31] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:02 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:03.080124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.080499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.080788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.086136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.100467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:03.101700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:03.135381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f5bb7-4cd7-4196-b33e-d40622fd3a43 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.176371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.177240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.177530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.177809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.189529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.191068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.197344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 18/32] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:03.198921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:12:03.211075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:03.211960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:03.212825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:12:03.212825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:12:03.212825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:12:03.212825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:03.215027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 15/33] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:03.216166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.315483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.318655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6d89e-ff6c-421b-902a-866227dd50b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.327200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba63d37-3520-4c18-9553-0f8d77cea745 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.355214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.363941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.365323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.365795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.366253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.367528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.369902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.374437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.375270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.375553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.375830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.377911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.379467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af140034-04f4-4c7f-adca-261f98831514 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.381241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 16/34] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:03.382511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909ca24-b4b6-4344-bd00-239603429aca None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.382802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909ca24-b4b6-4344-bd00-239603429aca None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.383084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909ca24-b4b6-4344-bd00-239603429aca None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.384551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 17/35] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:03.385054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909ca24-b4b6-4344-bd00-239603429aca None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.502922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 19/36] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jun 27 15:12:03 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:03.504270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.521289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909ca24-b4b6-4344-bd00-239603429aca None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.586333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613627-b741-48c0-a703-1670903dc123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.586928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613627-b741-48c0-a703-1670903dc123 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.587214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613627-b741-48c0-a703-1670903dc123 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.587495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36613627-b741-48c0-a703-1670903dc123 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.588264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36613627-b741-48c0-a703-1670903dc123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:03.588616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36613627-b741-48c0-a703-1670903dc123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.590080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36613627-b741-48c0-a703-1670903dc123 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.615330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 18/37] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:03.618459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613627-b741-48c0-a703-1670903dc123 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:03.675735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7020b-d548-4282-9de7-e3acc655f538 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.685158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.734045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.735166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.735654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.737408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.737948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.741296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdbcb9-755f-4b5b-b876-31472a8b8fd9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.767703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 20/38] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jun 27 15:12:03 2022] POST /identity/v3/roles => generated 223 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:03.770448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.770976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.771479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.803128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:03.806341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:03.811982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36613627-b741-48c0-a703-1670903dc123 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:03.871570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:03.872537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:03.872960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:03.873103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] PATH_INFO: `/v3/domains/default/users/d7a4d53e682240199113e94049a9ab62/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:03.875832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d7a4d53e682240199113e94049a9ab62, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:03.877354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ba0d7-c942-4d52-8125-4ac2138cabf0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:03.897422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:03.931895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 21/39] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:03.931895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:04.007427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 19/40] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Jun 27 15:12:03 2022] PUT /identity/v3/domains/default/users/d7a4d53e682240199113e94049a9ab62/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:04.009281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c6b6-f751-4e36-85da-53174a26930a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.009639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c6b6-f751-4e36-85da-53174a26930a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.009996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447c6b6-f751-4e36-85da-53174a26930a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.018841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 20/41] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:04.019812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447c6b6-f751-4e36-85da-53174a26930a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:04.060455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9abe38d-9264-4ecb-b79d-f451283c8dc1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:04.104247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:04.104958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.105238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.105531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.106256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:04.107777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34f23-cd58-42e0-9d51-4630ec6691d6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:04.152451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 22/42] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:12:03 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:04.152451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.152451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.152451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.161414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:04.178574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:04.180988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:04.206053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447c6b6-f751-4e36-85da-53174a26930a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:04.237232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:04.257032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:04.258103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.258570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.259025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.260050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:04.261978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:04.285998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 21/43] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jun 27 15:12:04 2022] POST /identity/v3/roles => generated 225 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:04.474866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 23/44] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:04.498725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:04.627354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915ab79-448a-48f5-9df5-449f34edaef5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:04.665434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:04.666258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.666551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.666832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.668190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:04.669778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14962acd-c52f-4768-8af9-9aff82a5dc52 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:04.671894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:04.677504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:04.785849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410a70c-5a73-454c-b8e8-ba052facc99f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:04.823672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:04.824398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:04.824692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:04.824975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:04.825708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:04.827229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e246ff-5b5f-49ce-8ad7-5aa68dbefbe4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:04.858610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:12:04 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:06.248398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d29f6-aae7-42e6-9feb-b9638ed3b8ce None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.248935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d29f6-aae7-42e6-9feb-b9638ed3b8ce None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.249335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d29f6-aae7-42e6-9feb-b9638ed3b8ce None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.251449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:06.260054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.260815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.261102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.283865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:06.285320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:06.298927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:06.367197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:06.391716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d29f6-aae7-42e6-9feb-b9638ed3b8ce None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:06.408707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:06.555466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d29f6-aae7-42e6-9feb-b9638ed3b8ce None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:06.621851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:06.623292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.623803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.624362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.647467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:06.663717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:06.665270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:06.666538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 24/49] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:06.675565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:06.856047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f8372-a958-4f1f-9a0a-b59bf2cf2da3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:06.915763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:06.916803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.917241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.917656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.918961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:06.919460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:06.921955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e78c6-d6c0-4cac-a6be-66618680305f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:06.936969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 26/50] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:06.943125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:06.970324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13d8-84b4-42e5-aefb-fff575493ef6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.970566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13d8-84b4-42e5-aefb-fff575493ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.970884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d13d8-84b4-42e5-aefb-fff575493ef6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.972343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 27/51] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:06.978843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:06.979133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:06.979416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:06.998120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:06.999420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:07.001954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:07.024135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:07.065958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5fa45-ca99-460e-ad8f-e99c1d35a1f2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:07.084366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 28/52] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:07.105050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:07.105973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:07.106326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:07.106562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:07.107841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:07.108004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:07.109692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:07.111129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 25/53] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:06 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:07.118958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:07.231558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d8252-012a-4ded-adc5-60039fd29c98 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:07.238248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd943-e9d3-432d-ac6a-79a5a3eb85c3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:07.269081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:07.269927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:07.270218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:07.270496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:07.271910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:07.273508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:07.275311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 29/54] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:07.279768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:07.281556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:07.281973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:07.282444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:07.282592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:07.284775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:07.287384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:07.290356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 26/55] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:07.294062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:07.299214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:07.303317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:07.440457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746720c-0b3d-40bb-a09d-c437f31cbf24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:07.505165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:07.506304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:07.506753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:07.507272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:07.508648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:07.511236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fb8fa-8be8-4473-9bce-e33a0565d9c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:07.536417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d179ef-1d0b-4ed3-aa43-499577d69f0e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:07.555985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 30/56] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jun 27 15:12:07 2022] POST /identity/v3/projects => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:12:07.580388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:07.581367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:07.581738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:07.582026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:07.582940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.domain.id=default, user.id=d7a4d53e682240199113e94049a9ab62) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:07.583257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:07.584903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf20a97-99ca-4768-847d-599b035708f1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:07.595851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 27/57] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jun 27 15:12:07 2022] GET /identity/v3/role_assignments?user.id=d7a4d53e682240199113e94049a9ab62&scope.domain.id=default&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 530 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:09.654986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbcb5-aaca-4117-8861-1a11896af9ad None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:09.655619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbcb5-aaca-4117-8861-1a11896af9ad None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:09.655619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbcb5-aaca-4117-8861-1a11896af9ad None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:09.657693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 31/58] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:09.663609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:09.664167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:09.664167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:09.682782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:09.684363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:09.751729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:09.760727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 28/59] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:09.786473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bbcb5-aaca-4117-8861-1a11896af9ad None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:09.853554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:09.902803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bbcb5-aaca-4117-8861-1a11896af9ad None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:09.943306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:09.944037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:09.944717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:09.944995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:09.946313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:09.947812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e0c5-d2e4-4592-a16d-f3b8893e9252 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:09.949530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 32/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:09.955864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:10.069873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b5068-2a04-467b-8868-ac819e1603e1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:10.106652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:10.107396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:10.107723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:10.108057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:10.109123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:10.110703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d241-de97-45bd-9e19-fcdced985bea None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:10.140115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 29/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:09 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:13.606561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afb7e9-2f4f-4a89-88a4-c3e8f2929233 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.607837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afb7e9-2f4f-4a89-88a4-c3e8f2929233 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.608556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afb7e9-2f4f-4a89-88a4-c3e8f2929233 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.610949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 33/62] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:13.617753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189ea90-b135-4f71-919e-b4cc929cd4d0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.619134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189ea90-b135-4f71-919e-b4cc929cd4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.619819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189ea90-b135-4f71-919e-b4cc929cd4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.629369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f7df4-56c5-45e5-ade7-32f5b9064e43 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.630271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f7df4-56c5-45e5-ade7-32f5b9064e43 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.631058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f7df4-56c5-45e5-ade7-32f5b9064e43 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.634360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 34/63] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:13.645544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.646524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.647267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.680514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c189ea90-b135-4f71-919e-b4cc929cd4d0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:13.683075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c189ea90-b135-4f71-919e-b4cc929cd4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:13.709428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:13.711514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:13.860471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:13.900764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 30/64] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:13.903212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff543d-bd36-4412-8dc9-c8bcab4e8016 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.904018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff543d-bd36-4412-8dc9-c8bcab4e8016 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.904812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff543d-bd36-4412-8dc9-c8bcab4e8016 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.908162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 31/65] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:13.925402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 35/66] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:13.926232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:13.927055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:13.927317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:13.927958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:13.978233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:13.980561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:14.038236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:14.127669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75734a-c8c9-44bd-b20b-39193f822436 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.151539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 32/67] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:14.151539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.228674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.230592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.231366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.232146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.259429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:14.276337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:14.277979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:14.279445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:14.283536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 36/68] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:14.284740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.350401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407f545-7ce0-4293-a30e-de76c7ec7020 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.375789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:14.427896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.429872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.430707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.431204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.432892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177d9f6-7690-4653-b6fa-124bd3171cbc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.454056 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:14.463315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:14.476603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.477779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.478251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.478718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:14.485218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:14.486113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:14.486852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:14.487548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 33/69] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:14.492569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.502803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:14.510875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:14.513365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 37/70] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:14.515571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.658240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551b2f-cea5-44c5-bed5-df8a2454764f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.693590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee69e-8448-44b2-bb0d-cd230ff5e026 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.703011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.703712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.704077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.704404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.709094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:14.710032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:14.712410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:14.726813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 34/71] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:14.728233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.754061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.755051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.755488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.755901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.757180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:14.757634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:14.759233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e9e3f-bbd5-46be-b4ab-c85f34ca7876 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:14.776403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 38/72] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:14.776403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f802e6c-6c31-4cbf-a433-f2612edda2a9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.776403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f802e6c-6c31-4cbf-a433-f2612edda2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.776403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f802e6c-6c31-4cbf-a433-f2612edda2a9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.781970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 39/73] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:14.781970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f802e6c-6c31-4cbf-a433-f2612edda2a9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:14.790048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:14.832673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:14.916797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f802e6c-6c31-4cbf-a433-f2612edda2a9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.921663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de610c-a60f-49a4-bf03-18f11bd06d32 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:14.973485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.974806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.975298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.975778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.978503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:14.979569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:14.980024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:14.980530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:14.981474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:14.981831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:14.984199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:14.998924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 35/74] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:15.000482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.001065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.003903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef070fa3-88f8-4ee5-9aef-cc8d68637525 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.006816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 40/75] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:15.008601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096711e-8527-427b-9a6d-be585bdc83f8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.008912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096711e-8527-427b-9a6d-be585bdc83f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.009346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096711e-8527-427b-9a6d-be585bdc83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.029827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8096711e-8527-427b-9a6d-be585bdc83f8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:15.031444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8096711e-8527-427b-9a6d-be585bdc83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:15.117939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 41/76] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:15.120399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae61a-2857-4d7c-b01d-3bd25de3a41f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.120871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae61a-2857-4d7c-b01d-3bd25de3a41f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.121357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae61a-2857-4d7c-b01d-3bd25de3a41f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.123991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 42/77] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:15.140258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ae61a-2857-4d7c-b01d-3bd25de3a41f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.145093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.157411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc615ab-97ef-46c9-9559-cc7ed2ec3049 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.163230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.206441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.207541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.208017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.208443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.209933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.211684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.213743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 36/78] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:14 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:15.215034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.268742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ae61a-2857-4d7c-b01d-3bd25de3a41f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.311885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.313064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.313451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.313765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.315222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.316883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac77a3c-4264-4608-a6f0-08b63a220987 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.318692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 43/79] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:15.321080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.321350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.321688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.346286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:15.346683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d6c8e-209b-4f1d-85f4-2856ba221ba5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.348122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:15.394548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.395384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.395785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.396217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.434495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.435405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 44/80] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:15.436616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.475799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.502426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.505299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.567279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2421593-b0e0-4329-a45d-d0f38ffe6fc2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.610801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.611757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.612042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.612419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.613921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.615474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.617290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 45/81] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:15.618115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.632132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 37/82] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:15.633085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:15.707141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.740233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.749541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459752a7-d36d-461d-8003-e3620491b0c2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.752071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c4211-09f9-4d35-99d0-8f0e7483e06f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:15.786779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.787805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.788269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.788757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.792214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:15.792924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:15.793955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:15.793955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:15.868475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.869994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.876464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:15.878005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:15.945275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:15.982045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.000211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 38/83] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:16.000211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.000580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 46/84] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:16.000580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.107418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a110c32-97f4-4fd0-9467-27bd92ca13bf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.111472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f73502-1677-4192-aff7-dbbf70274d45 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.144494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.145235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.145515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.145794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.149958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.150640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.150935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.151201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.151929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.153443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.157487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:16.165277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:16.166207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 39/85] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:16.167384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.186409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 47/86] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:15 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:16.188044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.221845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.260398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.329950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413531f3-eb14-412e-841e-b0ad55c215e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.345491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef27af-a786-45ae-a079-3a73d9e3e497 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.371911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055e606-8717-4179-9479-adf450799d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.372902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055e606-8717-4179-9479-adf450799d9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.373212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055e606-8717-4179-9479-adf450799d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.373458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055e606-8717-4179-9479-adf450799d9a None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.374890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055e606-8717-4179-9479-adf450799d9a None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.376508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055e606-8717-4179-9479-adf450799d9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.383396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 48/87] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.383396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055e606-8717-4179-9479-adf450799d9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.403419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.404661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.404955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.405276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.406638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.408126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.410255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 40/88] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.411340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.522304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055e606-8717-4179-9479-adf450799d9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.543067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.554026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.565181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6b1f-8c60-47b2-823d-f682d48f8854 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.572336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.573313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.573594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.573860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.575665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.577299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.580571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 49/89] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.581686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.606662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.607336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.607625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.607916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.609123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:16.609783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.611316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.621302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 41/90] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.622319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.707733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffba28c-32a8-4c41-a09b-9bab55f05437 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.745502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.746656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e60c5-9650-4fbc-8d23-a78153741a66 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:16.747321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.747321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.747321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.747962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:16.747962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.752356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.755748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 50/91] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.761339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.787886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:16.788642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:16.788935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:16.789215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:16.790873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:16.791667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:16.794336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:16.804646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 42/92] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:16.805773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:16.880530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.880777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:16.976264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf0d11-023c-4c00-8991-3c79d4eba8df None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:17.002604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ea629-d5dc-4b03-a050-4fde1c332b4f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:17.068198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:17.069540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:17.069827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:17.070108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:17.072121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:17.073903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:17.075706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 43/93] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:17.082793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:17.101448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:17.102139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:17.102423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:17.102694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:17.103554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=d7a4d53e682240199113e94049a9ab62) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:17.103872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:17.105433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:17.118452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 51/94] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:16 2022] GET /identity/v3/role_assignments?user.id=d7a4d53e682240199113e94049a9ab62&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:17.256185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:17.291412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a1a28-d9c6-4f74-9549-3c1be5f8a9a1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:17.366216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:17.367751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:17.368588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:17.369281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:17.521212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:17.524378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:17.701643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:17.741045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 44/95] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:12:17.746985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:17.936770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009558c-a21a-426c-af08-3e1debfbc1cc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:17.971410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:18.004584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:18.005492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:18.005781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:18.006064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] PATH_INFO: `/v3/projects/b373a0f0d2714268abb3c16850a81ac8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:18.007329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] RBAC: Authorizing `identity:get_project(project_id=b373a0f0d2714268abb3c16850a81ac8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:18.009065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193746f8-c97a-48f2-8d8e-efff0e776e6b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:18.010865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 52/96] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:17 2022] GET /identity/v3/projects/b373a0f0d2714268abb3c16850a81ac8 => generated 312 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:18.017210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:18.175547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f38e0-6efe-4833-a674-521b10af40ec None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:18.225215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:18.225924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:18.226212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:18.226493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:18.227366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=b373a0f0d2714268abb3c16850a81ac8, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:18.227684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:18.229217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31240ca9-cf0f-4caa-9f28-25e8b5992a65 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:18.235895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 45/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:18 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=b373a0f0d2714268abb3c16850a81ac8&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:20.046026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f60dc-8e59-49c4-9b47-de1544402f71 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.046737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f60dc-8e59-49c4-9b47-de1544402f71 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.046862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f60dc-8e59-49c4-9b47-de1544402f71 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.048483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 53/98] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:20.054359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.054841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.054941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.078053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:20.079300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:20.145695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:20.154286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a359-6d7d-4928-8281-97dc044cfae3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.154942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a359-6d7d-4928-8281-97dc044cfae3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.155456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a359-6d7d-4928-8281-97dc044cfae3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.157547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 54/99] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:20.164179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093335b-097d-4b0a-a518-6402906c8c89 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.164845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093335b-097d-4b0a-a518-6402906c8c89 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.165336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093335b-097d-4b0a-a518-6402906c8c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.178513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 46/100] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:20.190200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d093335b-097d-4b0a-a518-6402906c8c89 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:20.191706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d093335b-097d-4b0a-a518-6402906c8c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:20.255122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.285447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 55/101] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:20.295788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1b11-1b59-4c36-a2d9-6827891cf96d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.296685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1b11-1b59-4c36-a2d9-6827891cf96d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.297192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1b11-1b59-4c36-a2d9-6827891cf96d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.299656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 56/102] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:20.309103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.309427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.309958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.339858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:20.341464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:20.419916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 57/103] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:20.420547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.499902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041379f-5e68-4b92-a85a-57f0fbf7563b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:20.507835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:20.543703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:20.545210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.545570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.546134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.566100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:20.576450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f470617-7263-4cc2-8d50-9aa30efb0d70 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-feabec57-0c36-4197-9380-55fb11ac0832 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:20.582039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:20.582938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:20.583687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:20.584786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 47/104] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:20.587615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b7b5-c6fc-4814-887d-f0f54291ad94 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.587615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b7b5-c6fc-4814-887d-f0f54291ad94 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.587615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115b7b5-c6fc-4814-887d-f0f54291ad94 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.587615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 48/105] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:20.588216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115b7b5-c6fc-4814-887d-f0f54291ad94 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.620277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:20.621136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.621448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.621695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.634086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:20.642019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:20.644600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 58/106] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:20.645658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.651931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:20.751657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115b7b5-c6fc-4814-887d-f0f54291ad94 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:20.772388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:20.777377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51f0f2-a0c7-4bd3-80b5-77011b94d2e9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:20.795712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:20.796869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.797113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.797430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.810768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ad6cfe59-bbca-4b79-b706-c022c2221b0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:20.819405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:20.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:20.821397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 49/107] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:20.822208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31148-1cec-4df7-8627-02d12f8e6815 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.822208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31148-1cec-4df7-8627-02d12f8e6815 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.823036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d31148-1cec-4df7-8627-02d12f8e6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.837304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:20.838364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.838854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.839344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.840197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:20.840544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:20.841999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:20.846877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-57d31148-1cec-4df7-8627-02d12f8e6815 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:20.848769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-57d31148-1cec-4df7-8627-02d12f8e6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:20.857228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 59/108] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:20.857228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.964416 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 50/109] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:20.978774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977da-3c77-4028-8c4d-43680912a6dd None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:20.979138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977da-3c77-4028-8c4d-43680912a6dd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:20.979457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea977da-3c77-4028-8c4d-43680912a6dd None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:20.985976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 51/110] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:20 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:20.985976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea977da-3c77-4028-8c4d-43680912a6dd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:20.993358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480545b-45aa-4d91-be24-a0da877317d6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.025338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.039778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.040930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.041282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.041570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.042629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:21.043109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.044824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.055698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 60/111] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.056719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.089319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.140067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea977da-3c77-4028-8c4d-43680912a6dd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.181804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43fc21-5556-42db-9746-e9cadafc5fc1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.183358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.184214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.184529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.184914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.185815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:21.186252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.188794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de923fe6-c1ae-4703-9f7b-ee1ec06cc142 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.199763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 52/112] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.200850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.201138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.201423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.225192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:21.226436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:21.232762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.233904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.234524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.234937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.236777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.238760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.241369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 61/113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.242146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.336467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 53/114] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:21.338210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.372232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.383754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d2b6f-eaff-4fca-9391-22d1742ec8e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.386207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.424223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.425089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.425377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.425658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.426940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.428651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.430489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 62/115] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.431101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.519626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d558413-8920-481a-8b9c-a087bf2a9ef2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.557765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.558682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.558976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.559256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.560522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307009a-b502-45d1-9531-039eb47abc3f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.560649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.562328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.564254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 54/116] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.564885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.617161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.618462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.618876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.619283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] PATH_INFO: `/v3/users/b809a836dc0748c296b322433ef09e54` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.637423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] RBAC: Authorizing `identity:get_user(user_id=b809a836dc0748c296b322433ef09e54)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.639040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.641081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 63/117] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/users/b809a836dc0748c296b322433ef09e54 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.641966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.664394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.685477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.699473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a71926-b33d-4dc0-ab13-184ef579987e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.744912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.745974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.746319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.746660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.748234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.750059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.759888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 55/118] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:21.759888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.775397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92a21-528a-4422-8b25-3dc2c5896afa None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.813414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.814238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.814527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.814807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.827286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:21.836185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:21.837660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 64/119] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:21.839172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.882066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f72b0a-1533-4f18-9cc5-a2b71326653e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.925198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:21.925988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.926284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.926568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.927310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:21.928844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447efb7f-c158-4556-a811-b2986936d72c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:21.940434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:21.962585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 56/120] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:12:21 2022] POST /identity/v3/projects => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:21.964129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31fb-6498-4495-922f-0ff7d521be32 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:21.964477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31fb-6498-4495-922f-0ff7d521be32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:21.964786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31fb-6498-4495-922f-0ff7d521be32 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:21.966520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 57/121] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:21.970033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a31fb-6498-4495-922f-0ff7d521be32 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:21.977950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884e802-bd4f-4ac6-9fe7-2550eab37e0b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:21.991718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:22.035491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:22.036466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.036812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.037240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.038912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:22.040608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:22.042854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 65/122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:22.043581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:22.186270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a31fb-6498-4495-922f-0ff7d521be32 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:22.222633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f356c-e3fa-426a-bb68-666844c080c6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:22.252217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:22.253420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.253947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.254414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.269946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:22.272124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.272709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.273197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/users/0078685832a14fc8b0a4bbbb420b920b/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.276767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, user_id=0078685832a14fc8b0a4bbbb420b920b, role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:22.279077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:22.299016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 66/123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:22 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/users/0078685832a14fc8b0a4bbbb420b920b/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:22.300619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:22.317774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:22.430013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:22.434585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:22.461078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:22.477899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60337f5a-8df4-41c4-9ede-a0fe9a67b824 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:22.542354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:22.543706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.544075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.544488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.545362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:22.545745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:22.547708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f3d69-7cd4-4694-aa6b-715a79dede72 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:22.560448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 67/124] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:22.566616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.567100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.567554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.600293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:22.602493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:22.612555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 58/125] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:22.613820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:22.733039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:22.752354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 68/126] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:22.752354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:22.755699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ae8ab-48b4-47d3-9612-78a05fede1d5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:22.790708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:22.851170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:22.853622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:22.854134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:22.854632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/users/d7a4d53e682240199113e94049a9ab62/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:22.858571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, user_id=d7a4d53e682240199113e94049a9ab62, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:22.862556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:22.893441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 59/127] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:22 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/users/d7a4d53e682240199113e94049a9ab62/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:22.895500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.031656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ccb2-40f8-4e41-8a5d-bfaa892704fa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:23.191174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:23.191493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351d88d-e965-440f-a805-486e2f4c6a5d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:23.192855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:23.193363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:23.194213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:23.197299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:23.202806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:23.213067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 69/128] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:22 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:23.214273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.279897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:23.281610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:23.282188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:23.283077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:23.288141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:23.291740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:23.296030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 60/129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:22 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:23.298785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.346226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:23.400574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:23.487738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795edb2c-5ac6-4d42-b02e-de40e49f530e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:23.539318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16ecfc-bcd8-4bd7-aa3d-d13d392b68e7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:23.610135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:23.612071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:23.612813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:23.613630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:23.616046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:23.619186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:23.622935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 61/130] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:23 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:23.625129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.629638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:23.631081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:23.631571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:23.632070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:23.689373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:23.709131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:23.710069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:23.710848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 70/131] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 497 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:23.711482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.762067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cad5-14fb-46b7-b382-5ffad2f2e37a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:23.771975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:23.810579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:23.811808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:23.812359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:23.813016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:23.814323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=b809a836dc0748c296b322433ef09e54) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:23.814889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:23.816714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:23.824245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 62/132] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:23 2022] GET /identity/v3/role_assignments?user.id=b809a836dc0748c296b322433ef09e54&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:23.825516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:23.981722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112fef-4960-4110-9deb-6eaadd829db9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:24.005711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31113d49-e2cd-44a6-8bbc-bc22244218b1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:24.072712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:24.074048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:24.074683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:24.075493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:24.079566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:24.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:24.083369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:24.086652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:24.104158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:24.104158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:24.104158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:24.104158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/users/0078685832a14fc8b0a4bbbb420b920b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:24.106998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, user_id=0078685832a14fc8b0a4bbbb420b920b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:24.107860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:24.123515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 63/133] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:24.161482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 71/134] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:23 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/users/0078685832a14fc8b0a4bbbb420b920b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:24.176959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:24.255942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:24.409737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67942804-5805-43f6-ad83-4f041c37866c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:24.480137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:24.481720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:24.482216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:24.482725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:24.485011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:24.487909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:24.491323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 64/135] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:24 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:24.500923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:24.699596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597af969-dd9a-4c87-a573-92d26699590b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:24.776716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:24.778293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:24.778803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:24.779309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] PATH_INFO: `/v3/projects/b373a0f0d2714268abb3c16850a81ac8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:24.785938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] RBAC: Authorizing `identity:get_project(project_id=b373a0f0d2714268abb3c16850a81ac8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:24.789044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6f91-7483-45eb-9de4-7300d12618d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:24.792651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 72/136] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:24 2022] GET /identity/v3/projects/b373a0f0d2714268abb3c16850a81ac8 => generated 312 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:24.804997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:24.979655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:24.985203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e9df3-9c88-47a1-81bd-c4e53c100f24 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:25.033524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:25.034458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:25.034746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:25.035031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] PATH_INFO: `/v3/projects/b373a0f0d2714268abb3c16850a81ac8/users/0078685832a14fc8b0a4bbbb420b920b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:25.041303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] RBAC: Authorizing `identity:create_grant(project_id=b373a0f0d2714268abb3c16850a81ac8, user_id=0078685832a14fc8b0a4bbbb420b920b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:25.042838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77b244-8662-4bde-9098-ec6017d6f7ce None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:25.070960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 65/137] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:24 2022] PUT /identity/v3/projects/b373a0f0d2714268abb3c16850a81ac8/users/0078685832a14fc8b0a4bbbb420b920b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:25.766406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9cf02-987f-4a31-9585-42a870a45868 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:25.766941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9cf02-987f-4a31-9585-42a870a45868 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:25.767428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9cf02-987f-4a31-9585-42a870a45868 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:25.770888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 73/138] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:25.777168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac4a8-e5e9-480a-911c-600e730a4575 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:25.777742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac4a8-e5e9-480a-911c-600e730a4575 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:25.778171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac4a8-e5e9-480a-911c-600e730a4575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:25.818000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d5bac4a8-e5e9-480a-911c-600e730a4575 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:25.826288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d5bac4a8-e5e9-480a-911c-600e730a4575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:26.004802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 66/139] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:26.045108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9cf02-987f-4a31-9585-42a870a45868 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:26.139936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:26.206270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9cf02-987f-4a31-9585-42a870a45868 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:26.216162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a902cc-37f3-493a-877a-b0a001dfb54a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.216823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a902cc-37f3-493a-877a-b0a001dfb54a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.217318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a902cc-37f3-493a-877a-b0a001dfb54a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.219043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 67/140] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:26.238054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.238679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.239153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.265866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:26.266741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.267034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.267328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.294679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:26.296200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:26.299484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:26.307854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:26.308789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:26.309543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:26.310602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 74/141] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:26.321936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:26.501861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 68/142] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:26.535237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:26.566865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:26.623600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27b9c3-8b56-4e6c-ac52-82e8f32a5917 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:26.653291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:26.663643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b83d92-ac35-4545-8f31-e82087c9e261 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:26.707242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:26.708080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.708394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.708683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.709939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:26.711722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:26.713932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 69/143] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:26.719806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:26.732053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:26.733275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.734256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.735698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:26.736195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:26.738980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:26.753605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 75/144] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:26.773964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:26.827295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:26.843834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19da37a-4ae8-4e65-ab55-4e15f5ce3976 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:26.886001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:26.887082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:26.887531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:26.887975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:26.888920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:26.890707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:26.934029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 70/145] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:26 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:12:26.972953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7b34-5a20-4039-b25f-173119dd19a1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:27.034528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:27.035482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.035764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.036040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.037523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:27.039067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c12a7b-a13c-44df-bb7a-e931c9b6d014 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:27.040831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 76/146] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:26 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:27.046909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:27.250858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd9f0-5308-4942-a956-93708ff5581e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:27.306498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:27.307481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.307955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.308447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.332717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:27.393683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:27.395280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cc3d3-066d-4bd9-a861-284c733f00a5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:27.443784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c7bf8-a79d-41bf-89c5-2da4aa6e6349 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.444786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c7bf8-a79d-41bf-89c5-2da4aa6e6349 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.445323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c7bf8-a79d-41bf-89c5-2da4aa6e6349 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.446977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 77/147] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:27.461917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.462591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.463083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.493837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:27.500347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:27.552055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 71/148] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:27.553578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974aef-131f-4dbd-b11d-4309178c92c7 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.554036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974aef-131f-4dbd-b11d-4309178c92c7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.554464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a974aef-131f-4dbd-b11d-4309178c92c7 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.556030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 72/149] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:27.562793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.563113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.563374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.568070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:27.598117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:27.599345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:27.608694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 78/150] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:27.608694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:27.678149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:27.724928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 73/151] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:27.728332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:27.771393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b3811-a6b2-4123-8884-feb114b91234 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:27.817559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:27.818505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.818651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.819246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:27.821046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:27.822849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:27.825459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 79/152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:27 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:27.826430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:27.880220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:27.923283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba477d47-13c8-4fba-8a7b-f22826d94546 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:27.983349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:27.984707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:27.984995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:27.985266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.008379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08978c-66db-441d-9a56-fe7d47b7aecd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:28.013040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:28.021415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:28.022321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:28.023080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 74/153] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:28.024193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.062891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:28.063725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.064005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.064275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.089122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:28.097387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:28.098288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:28.099268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 80/154] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:28.101035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.123076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:28.275769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f830f3-9c6a-48f0-a5a7-d7a3e162e8cc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:28.278887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:28.281575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc9067-4849-4088-a887-291014a3bb29 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:28.327631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:28.328404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.328694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.328972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.329904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:28.330287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:28.332667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:28.340840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 75/155] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:28.342204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.381224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:28.381935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.382228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.382509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.383338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:28.383649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:28.385343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384c98b-50b5-4c51-9e35-052fda9c7dee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:28.399046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 81/156] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:28.401916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b3d1-4a2d-41d5-b885-5672a755e481 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.402211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b3d1-4a2d-41d5-b885-5672a755e481 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.402490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b3d1-4a2d-41d5-b885-5672a755e481 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.404350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 82/157] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:28.405242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b3d1-4a2d-41d5-b885-5672a755e481 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.523096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:28.530721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04182f2d-03be-43d1-9294-8fbfa26aeed8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:28.602945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:28.603892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.604181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.604486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.605270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:28.605582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:28.607102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f448544-864f-405b-8ba8-ba4c2ffa44bd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:28.623392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 76/158] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:28.625033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.625340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.625645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.654424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:28.655927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:28.669279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b3d1-4a2d-41d5-b885-5672a755e481 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:28.761019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:28.761936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:28.762221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:28.762492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] PATH_INFO: `/v3/users/d7a4d53e682240199113e94049a9ab62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:28.766617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 77/159] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:28.767376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.769317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a4d53e682240199113e94049a9ab62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:28.770788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:28.772591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 83/160] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/users/d7a4d53e682240199113e94049a9ab62 => generated 247 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:28.778326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:28.877888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:28.948784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:28.960025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4808ea2-ccdd-4516-9de7-ddf3b8ff0cd5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.006873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3582f30-5853-4403-ad15-11d0af4190a2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.041112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:29.042044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.042319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.042587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] PATH_INFO: `/v3/users/b809a836dc0748c296b322433ef09e54` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.043942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] RBAC: Authorizing `identity:get_user(user_id=b809a836dc0748c296b322433ef09e54)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:29.045485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:29.047842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 78/161] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/users/b809a836dc0748c296b322433ef09e54 => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:29.048697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:29.073695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:29.074372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.074658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.074938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.201454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:29.203411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d80eed-656a-451f-9a10-7ab615e6c719 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:29.250609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e5874-c8ba-4e5f-9064-08aed8162632 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.276714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:29.327419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:29.328814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.329147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.329421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.360878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:29.376789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:29.377722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:29.378479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 79/162] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:29.379115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:29.425814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 84/163] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:29.428068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:29.428902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86973-7806-4287-8b14-3b332eeced25 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.429205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86973-7806-4287-8b14-3b332eeced25 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.429482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86973-7806-4287-8b14-3b332eeced25 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.431474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 85/164] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:29.437475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86973-7806-4287-8b14-3b332eeced25 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:29.615064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29200cac-127b-4a6b-829b-56297c6e7448 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.632250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86973-7806-4287-8b14-3b332eeced25 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.687227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:29.688659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.689095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.689512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.691565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:29.694394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:29.700491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 80/165] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:29 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:29.702210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:29.711030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:29.721563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:29.722671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.723147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.723615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.724642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:29.725182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:29.726846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273ffe-acfe-4614-a7fe-07c03790b04b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:29.745032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 86/166] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:29.746992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:29.747524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:29.748067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:29.788563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:29.789850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:29.903850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:29.954350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1b1f-ee18-4a9c-a983-3a4a3a6d5b36 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:29.987364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 87/167] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:29.988867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.075641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:30.077258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.077765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.078291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.082366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:30.085084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:30.087907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 81/168] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:29 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:30.097259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.289263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469b9b3-41d6-437e-bafc-b32d1886e89c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:30.354198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:30.369711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bdcc0-b540-472b-a59a-9527f29cc307 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:30.372005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:30.372961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.373237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.373507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/users/b809a836dc0748c296b322433ef09e54/roles/1810bfbc8c644bfbbab1c652888b559f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.376767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, user_id=b809a836dc0748c296b322433ef09e54, role_id=1810bfbc8c644bfbbab1c652888b559f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:30.382918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:30.403431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 88/169] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:29 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/users/b809a836dc0748c296b322433ef09e54/roles/1810bfbc8c644bfbbab1c652888b559f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:30.405364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.459198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:30.460173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.460508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.460822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.465385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:30.466927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:30.469391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 82/170] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:30 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:30.470244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.513619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:30.649566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8bff-b3b8-424e-ad17-a8f4557769e0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:30.669359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52b5e2-dfbe-4873-943c-3e7089213774 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:30.715198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:30.715762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:30.716527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.716843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.717200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.717306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.717705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.718126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.719522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=d7a4d53e682240199113e94049a9ab62) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:30.720054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:30.722114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:30.730660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:30.733891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 83/171] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:30 2022] GET /identity/v3/role_assignments?user.id=d7a4d53e682240199113e94049a9ab62&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:30.735604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8cbcb2c5-1c13-4933-b7fb-08f812cb781a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:30.740504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:30.741545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 89/172] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:30.742543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de085ee-5a81-4e6f-ab24-5f24763657a9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.743365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de085ee-5a81-4e6f-ab24-5f24763657a9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.743365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de085ee-5a81-4e6f-ab24-5f24763657a9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.748492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 90/173] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:30.750604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:30.750892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:30.751160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:30.782200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:30.783455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:30.846552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:30.894284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 91/174] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:30.895660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:30.927784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:30.966624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681062-064b-4648-9a14-ac2826af785d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.012084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c9e37-9e37-45b8-8b67-cb2b9839a319 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.040914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.041592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.041881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.042153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.070927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.072216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.072783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.073480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.075138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:31.075758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:31.077709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de0856-48d9-4041-8afe-a81084a0559a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:31.098871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 92/175] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:31.102050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f3268-9326-4b3d-9750-7ab87a8d61af None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.102997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f3268-9326-4b3d-9750-7ab87a8d61af None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.103842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f3268-9326-4b3d-9750-7ab87a8d61af None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.107040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 93/176] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:31.113258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f3268-9326-4b3d-9750-7ab87a8d61af None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.137555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:31.139142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29876f0d-8d9e-4acc-ad91-bf105e1e335f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:31.185856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:31.251940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:31.277423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f3268-9326-4b3d-9750-7ab87a8d61af None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.279762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 84/177] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:31.283103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99fd9-c7b4-4a7f-b437-7e5d6bda9fa1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.283635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99fd9-c7b4-4a7f-b437-7e5d6bda9fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.284221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99fd9-c7b4-4a7f-b437-7e5d6bda9fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.317064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-38d99fd9-c7b4-4a7f-b437-7e5d6bda9fa1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:31.319494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-38d99fd9-c7b4-4a7f-b437-7e5d6bda9fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:31.324161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.325011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.325301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.325581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.338161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:31.346275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:31.348147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 94/178] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:31.349769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.452291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 85/179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:31.455048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159b06-621e-4bb1-be7b-a97ab0e0176b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.455559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159b06-621e-4bb1-be7b-a97ab0e0176b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.456065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75159b06-621e-4bb1-be7b-a97ab0e0176b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.457716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 86/180] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:31.458745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159b06-621e-4bb1-be7b-a97ab0e0176b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.474391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:31.481836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0c620-2780-4c5b-b352-944cccc32ef3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.527990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.528923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.529203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.529667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] PATH_INFO: `/v3/users/0078685832a14fc8b0a4bbbb420b920b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.531108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] RBAC: Authorizing `identity:get_user(user_id=0078685832a14fc8b0a4bbbb420b920b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:31.532774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:31.541830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 95/181] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/users/0078685832a14fc8b0a4bbbb420b920b => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:31.541830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.649457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75159b06-621e-4bb1-be7b-a97ab0e0176b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.677998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:31.696892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91733f-8acc-467c-9781-bda0129c23c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:31.709455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.710443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.710926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.711356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.713366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:31.715581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e7b82-298f-415a-9285-1ffb75a42abc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:31.725760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 87/182] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:31.725760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.725760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.725760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.745328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:31.746996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:31.760673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:31.761695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:31.762160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:31.762584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:31.763530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:31.763985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:31.765922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:31.777434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 96/183] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:31.783702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.821839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:31.877322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 88/184] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:31.878528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:31.966332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c0424-352c-48ee-922a-1c1dc3b9fb39 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.023422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.024766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.025217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.025634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.044673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:32.053026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007371bd-fefb-45b9-ba15-bed200bbc7da None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:32.059465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:32.060501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 97/185] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:32.069450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.079471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:32.104573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.105247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.105536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.105817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.210690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:32.225212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.227979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.228974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f7e12-c825-44c9-8c9c-45db9b8c5598 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.269426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.270100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.270395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.270948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.271801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:32.271958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.273541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.288375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 98/186] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:32.288375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.413734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 89/187] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:32.415426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.445952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:32.509362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc170-0d3b-4743-9fef-c136dd52f207 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.593107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.594161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.594470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.595111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.600441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3bf7-1694-43be-a1cd-07194276f725 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.615672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.617257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.619294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 99/188] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:32.620571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.633355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:32.679177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.680675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.681189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.681689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.683891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.687761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.691058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 90/189] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:32.692573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.768392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2894-983c-4e39-8b00-e5d1f871763e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.822480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:32.832548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.833624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.834060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.834347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.835790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:32.836292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.838924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.855223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 100/190] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:32.856108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.866079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433a575-8cb0-4417-a846-16e76f620e9b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:32.944640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:32.945947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:32.946353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:32.946749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] PATH_INFO: `/v3/projects/b373a0f0d2714268abb3c16850a81ac8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:32.948654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] RBAC: Authorizing `identity:get_project(project_id=b373a0f0d2714268abb3c16850a81ac8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:32.950728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:32.959895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 91/191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/projects/b373a0f0d2714268abb3c16850a81ac8 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:32.959895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:32.986983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:33.030454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b53f9d-a319-4704-91bd-ea0911fc038c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.067379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.068039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.068383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.068622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.083456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2e7f-e891-4535-a26d-a231aba507a8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.141381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.142129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.142403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.142680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.143425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:33.144967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:33.167751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:33.169319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:33.169724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 92/192] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:32 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:33.170701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:33.181908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:33.269989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:33.319719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 101/193] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:33.319719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:33.367681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d5f4-0948-451d-a998-9995ae0659b4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.419123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.420361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.420645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.420920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.423088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:33.424677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8db3-d4d5-470c-99f9-9d7bab01b53b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:33.426697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 93/194] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:33 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:33.427828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e4d2-667b-4f78-89cb-62f8ef1ff886 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.428115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e4d2-667b-4f78-89cb-62f8ef1ff886 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.428433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e4d2-667b-4f78-89cb-62f8ef1ff886 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.429859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 94/195] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:33.430708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e4d2-667b-4f78-89cb-62f8ef1ff886 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:33.505168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aef0-4711-4c7f-906c-1cf1abb11d4d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.545842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:33.562191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.563375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.563865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.564351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.566853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=b373a0f0d2714268abb3c16850a81ac8, user.id=0078685832a14fc8b0a4bbbb420b920b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:33.567410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:33.570618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:33.580012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 102/196] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:33 2022] GET /identity/v3/role_assignments?user.id=0078685832a14fc8b0a4bbbb420b920b&scope.project.id=b373a0f0d2714268abb3c16850a81ac8&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:33.581399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:33.582537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e4d2-667b-4f78-89cb-62f8ef1ff886 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.643394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.644887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.645443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.645795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.647310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:33.649009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41cadc-1203-40d2-aa63-6d52b69910ac None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:33.650234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:33.651220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 95/197] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:33 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:33.655205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.655205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.655205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.694108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:33.695515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:33.793292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39f5f-7917-4a24-9289-decb221afc20 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:33.860931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 96/198] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:33.862243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:33.870231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:33.872978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:33.875089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:33.875968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:33.972983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:34.041098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:34.042382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc42ca-58c2-4288-ac98-3145ff8e0e2a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:34.043684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:34.087796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:34.089822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:34.090347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:34.091091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:34.092167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=3ffa15f1d7f147aaa892f22eec9e21d2) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:34.092857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:34.095388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:34.103459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 97/199] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:33 2022] GET /identity/v3/role_assignments?user.id=3ffa15f1d7f147aaa892f22eec9e21d2&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:34.104673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:34.107246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:34.277439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 103/200] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:34.285270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f954fe5-581b-4094-b5fc-882c86903995 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:34.301262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:34.365912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:34.367232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:34.367706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:34.368184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:34.383492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:34.421621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:34.449144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:34.450533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:34.451742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 98/201] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:34.461628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:34.554371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d817062-af07-4366-a762-8f39f42bc39c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:34.633263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:34.634161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:34.634444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:34.634721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:34.636060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:34.639634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:34.645367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 104/202] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:34 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:34.652453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:34.658864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:34.681568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b64c91-8e41-4c6e-b33a-fec43b3dea44 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:34.754224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:34.755829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:34.756819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:34.759946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:34.761910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:34.762834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:34.765734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:34.783556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 99/203] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:34.793377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:34.896388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b87b0-f6a1-4b73-ae55-2c0a653b851b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:34.925405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:35.004193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745586a-3f49-402f-bb55-b014ca26fb67 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:35.030886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:35.031938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:35.032929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:35.032929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:35.034372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=3ffa15f1d7f147aaa892f22eec9e21d2) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:35.034899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:35.045941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:35.065086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 105/204] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:34 2022] GET /identity/v3/role_assignments?user.id=3ffa15f1d7f147aaa892f22eec9e21d2&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:35.072863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:35.074928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:35.075760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:35.076525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] PATH_INFO: `/v3/users/b809a836dc0748c296b322433ef09e54` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:35.079666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] RBAC: Authorizing `identity:get_user(user_id=b809a836dc0748c296b322433ef09e54)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:35.083050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:35.087195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 100/205] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:34 2022] GET /identity/v3/users/b809a836dc0748c296b322433ef09e54 => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:35.097173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:35.239278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:35.306640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aa5a0-4d58-4803-b973-ff02f9b683dd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:35.370240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:35.370936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:35.371222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:35.371501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:35.463821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:35.469666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:35.548839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:35.590111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 106/206] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:35.599352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:35.815792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e554af-71bc-4101-a3c4-75ede5c7de72 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:35.872740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:35.885381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:35.886312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:35.886600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:35.886877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:35.893299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:35.894829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca739b0-cd94-4822-ba5e-79a346a4f9a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:35.897070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 101/207] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:35 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:35.906420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:36.060654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79967d1c-fdb6-4347-b79e-bdd364aa81e7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:36.082128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19069c-8715-44e9-937b-a745e5dbe0a0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.082361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19069c-8715-44e9-937b-a745e5dbe0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.082675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19069c-8715-44e9-937b-a745e5dbe0a0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.084248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 102/208] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:36.089829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.090193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.090443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.116757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:36.117445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.117739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.118007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.118863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, scope.project.id=4a2d3aea849c46138a453442962777ab, user.id=b809a836dc0748c296b322433ef09e54) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:36.119181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:36.120733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aea2a0-9dee-4fe2-99b5-ab28a9eae8eb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:36.130479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:36.131728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:36.136803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 107/209] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:35 2022] GET /identity/v3/role_assignments?user.id=b809a836dc0748c296b322433ef09e54&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:36.242586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cdf-eee9-435a-829d-2b2aca80752e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.243536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cdf-eee9-435a-829d-2b2aca80752e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.244212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea17cdf-eee9-435a-829d-2b2aca80752e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.248653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 108/210] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:36.258974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.259898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.260682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.270027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 103/211] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:36.293577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:36.294869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:36.298261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:36.345063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:36.440968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 109/212] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:36.473446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcfb98-0105-44bf-bb5a-c172f51958ac None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:36.485625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:36.502114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:36.516597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:36.517431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.517715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.517975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.519679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:36.521594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:36.524452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 104/213] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:36.531526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:36.669383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2566-c401-493b-81b1-d790e14bdae6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:36.698379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d146e-b7c4-4d29-89c9-bbae29afc652 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:36.718390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:36.719509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.719817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.720090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.732574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:36.741065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:36.742776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:36.745619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 110/214] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:36.748069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:36.748069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.748069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.748069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.748819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:36.748819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:36.749072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:36.775742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:36.781492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 105/215] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:36 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:36.882735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befc205-bb80-4dea-bd80-ba8fd6f44330 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:36.923838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:36.924482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:36.924767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:36.925048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:36.925859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:36.926173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:36.927696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daa973-0f6e-4a3b-bbdc-d7711aa9f2c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:36.938159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 111/216] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:36.944383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:37.036692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:37.088950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd3be-6aa2-40dc-b800-586c4718bab8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.089741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd3be-6aa2-40dc-b800-586c4718bab8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.089741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd3be-6aa2-40dc-b800-586c4718bab8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.091262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 112/217] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:37.096899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.097168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.097446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.119701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:37.121080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:37.132588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc04327-7a97-4cbf-a88d-18a9fcfad4c1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:37.183309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:37.184273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.184649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.184897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] PATH_INFO: `/v3/users/42da144f8c10484bb3d530b79d4fb8f0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.203488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] RBAC: Authorizing `identity:get_user(user_id=42da144f8c10484bb3d530b79d4fb8f0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:37.205053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:37.207203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 106/218] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:36 2022] GET /identity/v3/users/42da144f8c10484bb3d530b79d4fb8f0 => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:37.213175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:37.270568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 113/219] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:37.296985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:37.362689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ae640-c3a7-436e-b07c-995a491b9f85 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:37.363363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:37.381591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:37.455432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:37.456518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.456977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.457421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.585427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:37.588172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:37.594494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae0242-bafe-46b8-93b5-5ea703a725eb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:37.658116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613762a-1009-446f-9201-8965a15246dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:37.659381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613762a-1009-446f-9201-8965a15246dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.659834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613762a-1009-446f-9201-8965a15246dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.660275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613762a-1009-446f-9201-8965a15246dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.713964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613762a-1009-446f-9201-8965a15246dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0613762a-1009-446f-9201-8965a15246dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:37.730424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:37.731393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:37.732190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:37.733431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 114/220] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:37.754123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613762a-1009-446f-9201-8965a15246dd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:37.789016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 107/221] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:12:37.795989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:37.810920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:37.934546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:37.946740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f831e86-ebe9-4c8d-b1f3-5c678cf6484f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:37.989896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:37.991053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.991465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:37.991877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:37.993367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:37.995184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a54f4-efea-44d9-8f8b-6307dca1089f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:37.997599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 108/222] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:37 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:37.999307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b126d-24ce-4df3-ab9e-28df55b346f3 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:37.999749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b126d-24ce-4df3-ab9e-28df55b346f3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.000166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b126d-24ce-4df3-ab9e-28df55b346f3 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.002034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613762a-1009-446f-9201-8965a15246dd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:38.002212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 109/223] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:38.003211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b126d-24ce-4df3-ab9e-28df55b346f3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:38.072344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:38.073666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.074163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.074769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.076239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:38.077211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:38.080179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bf7d5-0f93-460e-8b3f-e598918b1eff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:38.095475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:38.099707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 115/224] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:38.108449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.110085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.110728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.125641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b126d-24ce-4df3-ab9e-28df55b346f3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:38.138704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:38.141982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:38.166458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:38.167144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.167425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.167698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.168771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=42da144f8c10484bb3d530b79d4fb8f0) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:38.169045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:38.170696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:38.177882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 110/225] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:38 2022] GET /identity/v3/role_assignments?user.id=42da144f8c10484bb3d530b79d4fb8f0&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:38.178776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:38.236274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 116/226] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:38.295140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:38.406266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26ea04-6d00-44b8-9b00-734941f63901 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:38.442777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:38.491759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:38.493755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.494486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.495204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.502008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:38.505086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:38.511970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 111/227] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:38 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:38.521135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:38.554917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:38.597080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96962e94-2fd9-4ae3-b0e4-0a59d039827e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:38.704861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:38.706739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.708127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.745629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:38.768107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:38.769293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:38.770070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:38.778311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 117/228] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:38.788569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:38.875013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306602b1-9580-4466-a3a9-abec8cdb3a3f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:38.966267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:38.967917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:38.968574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:38.969087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:38.974170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:38.976497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:38.978733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 112/229] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:38 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:38.987748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:39.044041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef8bc-1553-4104-87f2-359c5def8abf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:39.104770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:39.142129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:39.146412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.146934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.147424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.148864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:39.149386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:39.153884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:39.177260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 118/230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:39.183553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:39.220764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:39.303708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b5e8-e7ee-49ba-a268-e7c15cb0139f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:39.431092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:39.431943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.432228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.432548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/3ffa15f1d7f147aaa892f22eec9e21d2/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.438089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, user_id=3ffa15f1d7f147aaa892f22eec9e21d2, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:39.439615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:39.463774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea5f12-83f0-44fb-8651-15a716fe8b4e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:39.475052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 113/231] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:38 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/3ffa15f1d7f147aaa892f22eec9e21d2/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:39.536011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:39.537648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.538314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.538989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.541952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:39.545480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e756ac-e78c-48d6-9bce-7ab7c0d1a273 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:39.549382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 119/232] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:39 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:39.555529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:39.577282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9060a-63ea-4776-878f-526b9aadc7b1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.577795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9060a-63ea-4776-878f-526b9aadc7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.578298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9060a-63ea-4776-878f-526b9aadc7b1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.581080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 120/233] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:39.598446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd3f-e55c-40a4-bba9-ad574557305a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.598953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd3f-e55c-40a4-bba9-ad574557305a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.599525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7cd3f-e55c-40a4-bba9-ad574557305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.646366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fcb7cd3f-e55c-40a4-bba9-ad574557305a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:39.648454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fcb7cd3f-e55c-40a4-bba9-ad574557305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:39.697263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7c11e-5348-455a-aa91-0e6f3d26de3b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:39.744728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:39.745772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.746059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.746338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.747726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:39.749429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e11f2-4db5-4b05-85e0-7575ff968dcd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:39.751331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 114/234] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:39 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:39.752810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0f53-a746-40ea-ba0a-b9fc29f31435 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.753100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0f53-a746-40ea-ba0a-b9fc29f31435 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.753379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b0f53-a746-40ea-ba0a-b9fc29f31435 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.755053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 115/235] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:39.758966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b0f53-a746-40ea-ba0a-b9fc29f31435 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:39.764858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:39.778540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:39.816267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 121/236] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:39.819250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79654404-cae7-4872-a242-c01dab0241cb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.819891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79654404-cae7-4872-a242-c01dab0241cb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.820544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79654404-cae7-4872-a242-c01dab0241cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.852899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-79654404-cae7-4872-a242-c01dab0241cb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:39.858922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-79654404-cae7-4872-a242-c01dab0241cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:39.919215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b0f53-a746-40ea-ba0a-b9fc29f31435 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:39.962815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 122/237] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:39.964387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79654404-cae7-4872-a242-c01dab0241cb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:39.970434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:39.971471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:39.971839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:39.972183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/3ffa15f1d7f147aaa892f22eec9e21d2/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:39.974648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, user_id=3ffa15f1d7f147aaa892f22eec9e21d2, role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:39.976255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:40.000394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 116/238] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:39 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/3ffa15f1d7f147aaa892f22eec9e21d2/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:40.010570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:40.179961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:40.231412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79654404-cae7-4872-a242-c01dab0241cb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:40.245297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532158f4-6f9d-4023-a31c-cb64f5c853b0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:40.252223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:40.284523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:40.285382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.285850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.286141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.296440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:40.297689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.298179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.298669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.300251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:40.302054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:40.303996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 117/239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:40.313129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:40.314047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:40.321943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:40.322895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:40.323899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 123/240] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:40.331048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:40.444723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe600c83-db1f-4a81-99e4-edc4eaaf7c3f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:40.478424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:40.485821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:40.486378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.486668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.486951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.487762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:40.489396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:40.516754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 118/241] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:40 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:12:40.573478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6e1f0-e314-4fd1-9d6c-e3a7d2b6de52 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:40.622098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:40.623241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.623817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.624383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.625407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:40.625930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:40.627600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e72d5-3543-4c01-a31a-692771e9c132 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:40.639618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 124/242] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:40.645661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:40.823895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f83dfe-75ed-4ff4-8bb1-7b038e180df5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:40.876070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:40.877027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.877312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.877588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] PATH_INFO: `/v3/users/05825404cb7c4db783b8921584e10e7f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.885753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aeea6-f715-47b7-b8b5-2cb1f2bb21fe None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.886385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aeea6-f715-47b7-b8b5-2cb1f2bb21fe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.886863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aeea6-f715-47b7-b8b5-2cb1f2bb21fe None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.892510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 125/243] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:40.895245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] RBAC: Authorizing `identity:get_user(user_id=05825404cb7c4db783b8921584e10e7f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:40.895796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:40.896287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:40.896809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:40.896935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:40.899131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 119/244] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:40 2022] GET /identity/v3/users/05825404cb7c4db783b8921584e10e7f => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:40.904790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:40.922250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:40.923587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:40.965591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:40.987864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:41.035590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b95a3-828d-469f-b47e-2533d7bb00bf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:41.096587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 126/245] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:41.123012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:41.124134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:41.124657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:41.125211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:41.128011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:41.311790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:41.315782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:41.390484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:41.400512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8754d3-a646-4dc2-9dbe-d0d907dda4e7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:41.494374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 120/246] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:12:41.502086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:41.533296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:41.534637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:41.544019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:41.544651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:41.601048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:41.619349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:41.620341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:41.621140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:41.628458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 127/247] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 497 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:41.644623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:41.720757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:41.802686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9d20-3d5c-4115-966d-b046003a1e67 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:41.840114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78eb5-ba79-4e6f-88d7-02d6c44aee17 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:41.892990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:41.894590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:41.895142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:41.895622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:41.896774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:41.897293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:41.898928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:41.906061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:41.907612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:41.908107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:41.908639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:41.914292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:41.914610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 128/248] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:41.916122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:41.921413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:41.924019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 121/249] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:41 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:41.930103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:41.964701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:42.108748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa3721-b731-4b09-826b-945c91567b9e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:42.155079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:42.156631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:42.157137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:42.157660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:42.158732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=05825404cb7c4db783b8921584e10e7f) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:42.159251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:42.161184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:42.169227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 122/250] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:41 2022] GET /identity/v3/role_assignments?user.id=05825404cb7c4db783b8921584e10e7f&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:42.216922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:42.255629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06871-19d8-4705-b5ee-50d754b7b063 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:42.349485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:42.350982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:42.351443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:42.351919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] PATH_INFO: `/v3/users/42da144f8c10484bb3d530b79d4fb8f0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:42.357703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] RBAC: Authorizing `identity:get_user(user_id=42da144f8c10484bb3d530b79d4fb8f0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:42.361078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:42.364345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 129/251] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:41 2022] GET /identity/v3/users/42da144f8c10484bb3d530b79d4fb8f0 => generated 297 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:42.370607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:42.488906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:42.518955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343ce4-2ba3-4144-9d04-5c41654ce759 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:42.570809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:42.572401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:42.572919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:42.573432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:42.575711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:42.579139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb3fdc-e27e-4ebf-8f4a-2f9e1309d217 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:42.582596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 123/252] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:42 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:42.589443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:42.807012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb33b-1b4a-49bf-97d9-ac3ed33b53a7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:42.905440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:42.907134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:42.907633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:42.908202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/42da144f8c10484bb3d530b79d4fb8f0/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:42.914756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, user_id=42da144f8c10484bb3d530b79d4fb8f0, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:42.917545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b34ff-a8ad-43b5-955d-5ce2503e0e88 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:42.951216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 130/253] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:42 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/42da144f8c10484bb3d530b79d4fb8f0/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:43.024431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb1fe-ba32-4480-b377-4b952fbe9758 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.024961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb1fe-ba32-4480-b377-4b952fbe9758 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.025496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb1fe-ba32-4480-b377-4b952fbe9758 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.028675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 124/254] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:43.038540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.038993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.039449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.070899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:43.072992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:43.095773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:43.105421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222163-933e-473c-b2fb-0b60918658ef None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.105842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222163-933e-473c-b2fb-0b60918658ef None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.106282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38222163-933e-473c-b2fb-0b60918658ef None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.109708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 125/255] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:43.119103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.119556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.120066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.162283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:43.164168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:43.257016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 131/256] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:43.323124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:43.334947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 126/257] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:43.358898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:43.604724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:43.631075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235761-8439-43dd-b659-fbb48be8a7cc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:43.648211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca3a2d-e8a1-49bd-a08e-38794ea1ede1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:43.705220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:43.706492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.706959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.707413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.719548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:43.721663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.722420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.723180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.733097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:43.744351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:43.745717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:43.747018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 127/258] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:43.750171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:43.758108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:43.781918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:43.783690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:12:43.785294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:43.794755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 132/259] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:12:43.800478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:43.825116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:43.915265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3277f-168a-4abe-8d0f-ac2132bc2dcf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:43.994317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:43.995039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:43.995317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:43.995600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:43.996431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:43.996747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:43.998209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:44.000218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:44.026689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 128/260] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:44.036758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:44.150781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51329f-8738-40c6-a7e7-e97a6cca9aff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:44.261301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:44.262025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.262312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.262573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.263379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:44.263694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:44.265269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:44.283183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 133/261] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:44.291108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:44.337638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f008316-7ec6-4067-bfc4-e167e1545f29 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:44.406753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:44.408244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.408788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.409286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.411741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:44.414594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:44.418324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 129/262] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:44 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:44.427535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:44.504967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:44.537069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:44.581776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a49989-10c5-444b-8367-0da19b0a89f7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:44.620829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:44.621511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.621816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.622080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.634445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277b9d0-0161-44e2-b22d-53a42414ab6a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:44.700374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:44.701926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5a345-2bce-48c4-b4b0-57566e428d0b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:44.774153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:44.777216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:44.778168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.778457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.778740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] PATH_INFO: `/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.780464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] RBAC: Authorizing `identity:get_user(user_id=3ffa15f1d7f147aaa892f22eec9e21d2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:44.781938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ecc3f8-b385-4fc8-b214-f0f4d32aae64 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:44.783786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 134/263] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:44 2022] GET /identity/v3/users/3ffa15f1d7f147aaa892f22eec9e21d2 => generated 283 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:44.789905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06d44-c192-4807-94b4-89bff3cb953b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.790179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06d44-c192-4807-94b4-89bff3cb953b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.790485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06d44-c192-4807-94b4-89bff3cb953b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.793623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 135/264] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:44.794561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b06d44-c192-4807-94b4-89bff3cb953b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:44.815924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 130/265] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:12:44.817432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:44.817714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:44.817996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:44.837531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:44.838810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:44.915439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 131/266] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:44.916062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:45.035509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:45.043612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096ad50-27e4-487a-afd0-a011feb49a1f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:45.082117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:45.083084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.083367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.083688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.085002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:45.086501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:45.088611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 132/267] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:44 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:45.089439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:45.181612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b06d44-c192-4807-94b4-89bff3cb953b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:45.203755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264c4c-df6d-47b6-86a9-4c7b81217c3f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:45.245229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:45.246061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.246352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.246630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.247897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:45.249502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:45.251230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 133/268] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:45.252026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:45.291816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:45.325749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:45.327108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.327302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.327718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.342496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:45.373972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7bcf5-31b7-473d-a49a-707550796f8a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:45.412305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:45.413413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.413905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.414379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.415455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=3ffa15f1d7f147aaa892f22eec9e21d2) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:45.415994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:45.417761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:45.424998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 134/269] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:45 2022] GET /identity/v3/role_assignments?user.id=3ffa15f1d7f147aaa892f22eec9e21d2&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:45.426171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:45.534754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:45.542970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454027-558f-4360-8d21-610228ec7ce5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:45.582344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:45.582927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:45.583719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.584245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.584779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.585537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:45.585957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:45.587820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51e587-fa12-41b6-b082-98cb29ad2b80 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:45.620937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 135/270] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:12:45 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:45.785124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc19d1-4a7c-494a-a7ae-08ecafb62522 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.785891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc19d1-4a7c-494a-a7ae-08ecafb62522 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.786389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc19d1-4a7c-494a-a7ae-08ecafb62522 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.788149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 136/271] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:45.794301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:45.794885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:45.795373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:45.814765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:45.816552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:45.818197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 136/272] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1024 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:45.828702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:45.912729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:45.919345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 137/273] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:45.944936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.063820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:46.083589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c53fff-7f66-4f13-b80c-eea7c9170c86 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.089846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab7bdb-9332-4842-a9dd-b3bf8826fcc1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.127093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:46.128036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.128430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.128776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.142549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:46.151856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:46.153165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:46.154404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 138/274] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:46.161045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.170952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:46.173375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.173861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.174383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.178013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:46.181746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:46.185817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 137/275] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:45 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:46.197425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.308293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03bb07-7ac8-4a3a-85ee-342d1ec053e5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.353908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:46.355024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.355319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.355595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.356573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:46.357002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:46.359107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7bec7-32fc-456b-8427-539d19d776f9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:46.378083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 139/276] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:46.386838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9714c-033b-40eb-be24-f530dcec588b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.387872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725698-cf29-4680-b88e-9f610868bfd8 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.388724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725698-cf29-4680-b88e-9f610868bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.389058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725698-cf29-4680-b88e-9f610868bfd8 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.391291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:46.393203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 140/277] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:46 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:46.394721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725698-cf29-4680-b88e-9f610868bfd8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.403145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:46.436417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:46.437390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.437838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.438254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.439387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=3ffa15f1d7f147aaa892f22eec9e21d2) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:46.439869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:46.441754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64ffa4-e15e-4b97-96c3-3453c260525b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:46.449456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 138/278] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:46 2022] GET /identity/v3/role_assignments?user.id=3ffa15f1d7f147aaa892f22eec9e21d2&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:46.451038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ff13-d7fc-4be2-870b-89bc4e751a47 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.451495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ff13-d7fc-4be2-870b-89bc4e751a47 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.452069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98ff13-d7fc-4be2-870b-89bc4e751a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.474976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-df98ff13-d7fc-4be2-870b-89bc4e751a47 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:46.476632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-df98ff13-d7fc-4be2-870b-89bc4e751a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:46.647096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725698-cf29-4680-b88e-9f610868bfd8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.690600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 139/279] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:46.693256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b2f-77d6-4325-abfb-e63ac63ac4a8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.694315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b2f-77d6-4325-abfb-e63ac63ac4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.694986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5455b2f-77d6-4325-abfb-e63ac63ac4a8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.697660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 140/280] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:46.707043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.707820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.708494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.724379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:46.725839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:46.726295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:46.726740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] PATH_INFO: `/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:46.748700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:46.750848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:46.758811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] RBAC: Authorizing `identity:get_user(user_id=a04ef61fdd0a4c9ebdd8ccf389b73656)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:46.762256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:46.766163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 141/281] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:46 2022] GET /identity/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656 => generated 293 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:46.777579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.835561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 141/282] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:46.841527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:46.890269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:46.920909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:46.982486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dc841-b7a9-4fac-8c23-48d948b44363 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:46.989339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe816c1-c167-41bc-982d-4b7f3d6a4963 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:47.023694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.024821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.025275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.025777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.026808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, user.id=a04ef61fdd0a4c9ebdd8ccf389b73656, scope.system=all) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:47.027295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:47.029086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:47.035587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 142/283] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jun 27 15:12:46 2022] GET /identity/v3/role_assignments?user.id=a04ef61fdd0a4c9ebdd8ccf389b73656&scope.system=all&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:47.041004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:47.072271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.073706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.074219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.074657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.120268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:47.144677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:47.145650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:47.146418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:47.147780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 142/284] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:47.157611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:47.318198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:47.337840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd1ae2-a9d9-4a77-9ff8-5fdf6c798d94 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:47.397955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e5766-2b31-4cc7-9cb1-1a17dc5cfca9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:47.444266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.445138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.445423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.445697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.451879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:47.485415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:47.502757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.504267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.505063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.505789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.507363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:47.508200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:47.511152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:47.517177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:47.518845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:47.521022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:47.530652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 143/285] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 494 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:47.537740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 143/286] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:47.546596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:47.558077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:47.828371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562964b8-20f5-40ca-ae86-253817ab458e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:47.841831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20078-1b34-40e0-ba0d-56557fc73e74 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:47.915411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.916330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.916622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.916952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] PATH_INFO: `/v3/users/05825404cb7c4db783b8921584e10e7f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.918217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] RBAC: Authorizing `identity:get_user(user_id=05825404cb7c4db783b8921584e10e7f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:47.919736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:47.921497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:47.921911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 144/287] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/users/05825404cb7c4db783b8921584e10e7f => generated 297 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:47.922622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:47.923107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:47.923596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:47.936353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:47.936353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:47.936353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:47.944675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:47.951536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 144/288] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:47.961991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:48.024990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:48.085677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:48.230768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e0bfc-df4e-4a30-b4a0-e764a95c0747 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:48.258777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7d21-9416-481a-b4d0-cacb61bd2569 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:48.298614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:48.300680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.301362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.302024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] PATH_INFO: `/v3/users/42da144f8c10484bb3d530b79d4fb8f0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.304371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] RBAC: Authorizing `identity:get_user(user_id=42da144f8c10484bb3d530b79d4fb8f0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:48.307150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:48.310803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/users/42da144f8c10484bb3d530b79d4fb8f0 => generated 297 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:48.317031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:48.363450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:48.365505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.366132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.366602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.368470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:48.372643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:48.377991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 145/290] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:47 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:48.385465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:48.537595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:48.558928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bf861-6737-4842-b0d0-3df862e39d85 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:48.580692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387f3d-8883-47fb-9755-966ae14f1cf8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:48.599585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:48.600285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.600596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.600873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.621012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:48.655164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:48.657213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.657933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.658688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/05825404cb7c4db783b8921584e10e7f/roles/1810bfbc8c644bfbbab1c652888b559f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.662949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, user_id=05825404cb7c4db783b8921584e10e7f, role_id=1810bfbc8c644bfbbab1c652888b559f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:48.666063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079065d2-e48e-44f2-b887-8d75cbdcbd50 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:48.691050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:48.693019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 146/291] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:12:48 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/users/05825404cb7c4db783b8921584e10e7f/roles/1810bfbc8c644bfbbab1c652888b559f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:48.694717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:48.695862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3335241-b09f-4ea6-a65e-1468c82960c0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.696554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3335241-b09f-4ea6-a65e-1468c82960c0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.697140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3335241-b09f-4ea6-a65e-1468c82960c0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.700242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 147/292] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:48.708845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:48.709394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:48.709946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:48.745242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:48.747746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:48.879826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 148/293] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:48.886805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:48.920695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 146/294] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:48.926439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:48.937523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.121625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a10fb-834b-4e70-a938-6a128d084d88 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.122873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a7f4d-c7e1-4854-8795-47c9e3f17de8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.134239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:49.165458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.166820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.166820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.167123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.168249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:49.169798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:49.172004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 149/295] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:48 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:49.177584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.209237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.211006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.211644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.212258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.252358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:49.273282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:49.275146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:49.276896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:49.290313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 147/296] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:49.303842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.304540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4434e82-621e-4138-9f10-8a518da7732f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.344465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.345260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.345603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.345946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.346878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=42da144f8c10484bb3d530b79d4fb8f0) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:49.347279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:49.349165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7317b8-c801-4132-8acf-d648a5c91cb3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:49.356496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 150/297] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:49 2022] GET /identity/v3/role_assignments?user.id=42da144f8c10484bb3d530b79d4fb8f0&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:49.421871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:49.455777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b051-82b3-477d-86d0-97b0077994c1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.456171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b051-82b3-477d-86d0-97b0077994c1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.456551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b051-82b3-477d-86d0-97b0077994c1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.458316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 151/298] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:49.468723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.469110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.469393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.498816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:49.500065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:49.518560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ea730-4501-4bba-81c0-9ddc76a4b277 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.597262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:49.610466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.611988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.612733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.613423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.615040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:49.615835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:49.628392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:49.630860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 152/299] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:49.651277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 148/300] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:49.672887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.702769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.891721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e26ef9-545f-46ad-b8e7-392025e5caf2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.911384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d931d-5ec8-4f4f-b283-dc2dd4d1b037 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:49.915442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:49.959136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.960548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.960717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:49.961038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.961604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] PATH_INFO: `/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.962732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.963459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.964127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.966566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:49.969807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335eedf-98d7-47ab-9c27-42cdd7a6388b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:49.980902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 149/301] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:49.982537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ffc4-a64a-475d-b4fd-9d8e347dd562 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.982837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ffc4-a64a-475d-b4fd-9d8e347dd562 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.983740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4ffc4-a64a-475d-b4fd-9d8e347dd562 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:49.985848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 150/302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:49.985848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4ffc4-a64a-475d-b4fd-9d8e347dd562 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:49.989725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] RBAC: Authorizing `identity:get_user(user_id=a5362a3aa9b444ecb2e3a183a7573e0b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:49.992054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf7d91-611e-453b-9445-c8310ff81d8e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:49.995394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 153/303] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:49 2022] GET /identity/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b => generated 293 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:49.997605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:49.998087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:49.998707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.027092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:50.029106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:50.070757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:50.149586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4ffc4-a64a-475d-b4fd-9d8e347dd562 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:50.158183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 154/304] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:50.164048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:50.211732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:50.212723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:50.213197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:50.213662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] PATH_INFO: `/v3/groups` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.214854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:50.217636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:50.243287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 151/305] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jun 27 15:12:49 2022] POST /identity/v3/groups => generated 229 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:50.250952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:50.322798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:50.364294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6401-67ca-4778-b0ba-b263ad334905 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:50.438072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:50.438944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:50.439348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:50.439756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.441168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, user.id=a5362a3aa9b444ecb2e3a183a7573e0b, scope.system=all) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:50.441696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:50.444732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:50.455957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 155/306] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jun 27 15:12:50 2022] GET /identity/v3/role_assignments?user.id=a5362a3aa9b444ecb2e3a183a7573e0b&scope.system=all&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:50.480746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d401650-f742-41f3-807c-49af6d6bb811 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:50.540766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:50.541617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:50.541907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:50.542176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.555768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:50.564062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:50.566292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:50.567784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 152/307] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:50.574831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:50.610883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:50.704253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9305a-e3ad-4a31-ae77-a52c6b2fe2d3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:50.751499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:50.753199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:50.753968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:50.754721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.756361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:50.757218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:50.759812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:50.775862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 156/308] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:50.786178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:50.927671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:50.939481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00783338-7624-4b2b-b54a-d4a81b75b7d7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:50.986881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:50.987790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:50.988068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:50.988365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] PATH_INFO: `/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:50.991148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] RBAC: Authorizing `identity:get_user(user_id=a04ef61fdd0a4c9ebdd8ccf389b73656)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:50.992894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6eea-b692-4380-a332-707597f354fd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:50.995833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 153/309] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:50 2022] GET /identity/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:51.001896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:51.104502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:51.121780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093adf0b-6e3d-47a7-afb3-93a4e9670b1e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:51.180166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:51.181089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:51.181370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:51.181645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] PATH_INFO: `/v3/system/users/a04ef61fdd0a4c9ebdd8ccf389b73656/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:51.183210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a04ef61fdd0a4c9ebdd8ccf389b73656, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:51.184706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31566052-d19a-42e7-8ac7-4848377d0b85 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:51.197322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 157/310] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jun 27 15:12:51 2022] PUT /identity/v3/system/users/a04ef61fdd0a4c9ebdd8ccf389b73656/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:51.901857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d112-35a4-4ad5-8147-bc99a172d372 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:51.902350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d112-35a4-4ad5-8147-bc99a172d372 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:51.902845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957d112-35a4-4ad5-8147-bc99a172d372 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:51.905478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 154/311] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:51.914810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:51.915313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:51.915774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:51.945127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:51.947201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:52.080505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 158/312] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:52.115854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957d112-35a4-4ad5-8147-bc99a172d372 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:52.313808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957d112-35a4-4ad5-8147-bc99a172d372 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:52.373881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:52.375217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.375728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.376235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.391106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-79e938e1-0e51-4443-b682-cab4554953b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:52.406943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:52.407904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:52.408716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:52.413645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 155/313] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:52.419671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:52.575635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:52.617505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505b1c1-f4ce-4a5c-a30e-059cb14e33ce None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:52.636334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252144-82a5-4c8d-92ec-7790461dac3a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.636665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252144-82a5-4c8d-92ec-7790461dac3a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.636920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93252144-82a5-4c8d-92ec-7790461dac3a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.638356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 156/314] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:12:52.644176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.644629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.645005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.671680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:52.672408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.672691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.672961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.673726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:52.674032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:52.675456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc227735-4574-4458-b92b-e36a7f0584e7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:52.678599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:52.680670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:52.685097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 159/315] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:52.686471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6219-055f-4e65-a1e4-bce3739d1e46 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.686855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6219-055f-4e65-a1e4-bce3739d1e46 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.687140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b6219-055f-4e65-a1e4-bce3739d1e46 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.688832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 160/316] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:52.695134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:52.695452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:52.695698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:52.713197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:52.714486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:52.715014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:52.785043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 157/317] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:52.786410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:52.804356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 161/318] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:52.810097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:52.943215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db012be-af3e-4b7b-9334-392b00f085e0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:52.943853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:52.991407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df7dfe-668e-40de-9834-dd4dace2127b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.010624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.012193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.012745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.013242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] PATH_INFO: `/v3/users/05825404cb7c4db783b8921584e10e7f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.015592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] RBAC: Authorizing `identity:get_user(user_id=05825404cb7c4db783b8921584e10e7f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.018442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.021686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 158/319] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:52 2022] GET /identity/v3/users/05825404cb7c4db783b8921584e10e7f => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:53.023270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.042429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.043837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.044243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.044697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.058027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:53.065356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:53.079179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:53.081260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:53.082888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 162/320] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:53.085053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.196834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe80442-9439-4630-a5c7-c9e83e10e691 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.255259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.257215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.258071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.258818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.261525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.264767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.266558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016895-28e2-401c-be60-0033fb6a020a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.268677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 159/321] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:12:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:53.270491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.316574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:53.327227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.327911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.328197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.328528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.423412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:53.439889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0708-8bac-431a-ae7d-84df9494ccfd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.462992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.465686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.482567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.483317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.483592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.483874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.484705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:53.485017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.486487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.503675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 160/322] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:53.503675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.615297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 163/323] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:53.617070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.658580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:53.659209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985ca82-2925-4780-a6a9-24b003423103 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.707243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.707935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.708236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.708537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] PATH_INFO: `/v3/groups` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.709258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.710804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132a348-84a5-47bd-9bd2-58dca21b6e3c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.729796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 161/324] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jun 27 15:12:53 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:12:53.731906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745ca0-7f01-485c-99cd-3e88ea4d4dc6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.732425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745ca0-7f01-485c-99cd-3e88ea4d4dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.732979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745ca0-7f01-485c-99cd-3e88ea4d4dc6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.735323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 162/325] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:53.736635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5745ca0-7f01-485c-99cd-3e88ea4d4dc6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:53.756426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:53.778900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2e87c-c369-418c-b6a5-a3a8cca73a5f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.831625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.832939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.833339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.833680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] PATH_INFO: `/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.835484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] RBAC: Authorizing `identity:get_user(user_id=a5362a3aa9b444ecb2e3a183a7573e0b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.837230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e9594-24e7-4d7c-a57a-0bd66ff4085e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.839227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 164/326] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:53 2022] GET /identity/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:53.844590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.844590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.844590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.879906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:53.882264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:53.923416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5745ca0-7f01-485c-99cd-3e88ea4d4dc6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:53.979612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:53.981231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:53.981674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:53.982159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:53.985092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:53.988819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:53.992219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 163/327] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:53 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:53.993687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:54.007018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 165/328] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:54.007018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:54.066498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:54.135994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:54.169636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94c245-e361-4048-bc8e-08856fcce426 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:54.198252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ad9c1-db80-4c78-b6d3-76362f277852 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:54.245183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:54.245956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:54.248095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:54.248095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] PATH_INFO: `/v3/system/users/a5362a3aa9b444ecb2e3a183a7573e0b/roles/1810bfbc8c644bfbbab1c652888b559f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:54.249137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a5362a3aa9b444ecb2e3a183a7573e0b, role_id=1810bfbc8c644bfbbab1c652888b559f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:54.250590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:54.263057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:54.263676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:54.264130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:54.264452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:54.265259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33, user.id=05825404cb7c4db783b8921584e10e7f) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:54.265601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:54.270057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 164/329] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jun 27 15:12:53 2022] PUT /identity/v3/system/users/a5362a3aa9b444ecb2e3a183a7573e0b/roles/1810bfbc8c644bfbbab1c652888b559f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:12:54.270057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:54.271404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:54.279594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 166/330] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:12:54 2022] GET /identity/v3/role_assignments?user.id=05825404cb7c4db783b8921584e10e7f&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:54.477934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85878f87-6c98-473c-b3e3-2a356f2ee30a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:54.554649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:54.555817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:54.556201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:54.558811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:54.572126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:54.578896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:54.595410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:54.596403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:54.597211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:54.600282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 165/331] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:54.606953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:54.798908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:54.817823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3addc7-e04b-4c20-978f-290f0a609819 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:54.887242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:54.888421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:54.888921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:54.889459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:54.890941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:54.891497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:54.894183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:54.911185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 167/332] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:12:54.917583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:55.069527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bb62-9008-401e-a308-010867f6bc80 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:55.120517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:55.121808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:55.122318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:55.122816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] PATH_INFO: `/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:55.124534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] RBAC: Authorizing `identity:get_user(user_id=a04ef61fdd0a4c9ebdd8ccf389b73656)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:55.126346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9992-9dfa-4cad-8432-16fe9bcc52d9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:55.128693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 166/333] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:54 2022] GET /identity/v3/users/a04ef61fdd0a4c9ebdd8ccf389b73656 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:55.134923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:55.316478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585a142-a7d7-484f-a727-9b4ca4ee376f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:55.383370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:55.384446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:55.384942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:55.385426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:55.387068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] RBAC: Adding query filter params (role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, user.id=a04ef61fdd0a4c9ebdd8ccf389b73656, scope.system=all) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:55.387607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:55.390504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad803-724e-44db-804e-f5ec00fe20cb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:55.418456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 168/334] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jun 27 15:12:55 2022] GET /identity/v3/role_assignments?user.id=a04ef61fdd0a4c9ebdd8ccf389b73656&scope.system=all&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:57.618872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48590-dd10-4d20-8db9-2e0e4e4ffd2a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.619942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48590-dd10-4d20-8db9-2e0e4e4ffd2a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.620505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a48590-dd10-4d20-8db9-2e0e4e4ffd2a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.622609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 167/335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:57.630405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d88b8-f210-4989-ac04-5b26466a5912 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.631056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d88b8-f210-4989-ac04-5b26466a5912 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.631056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d88b8-f210-4989-ac04-5b26466a5912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.644551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6f28a-0932-44fc-9b72-4e0a4f28babc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.645201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6f28a-0932-44fc-9b72-4e0a4f28babc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.645708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6f28a-0932-44fc-9b72-4e0a4f28babc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.647703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 168/336] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:57.653802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.654516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.655230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.662798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0f7d88b8-f210-4989-ac04-5b26466a5912 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:57.665816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0f7d88b8-f210-4989-ac04-5b26466a5912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:57.693922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:57.695534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:57.732727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:57.809145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 169/337] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:57.814552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc166-c498-4ed4-b0f4-4df0c0108b47 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.814840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc166-c498-4ed4-b0f4-4df0c0108b47 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.815126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cc166-c498-4ed4-b0f4-4df0c0108b47 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.817156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 170/338] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:57.823599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:57.823892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:57.824175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:57.852933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:57.857592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:57.858913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 169/339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:12:57.859138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:57.872139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:57.958591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 171/340] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:57.959749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.074192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:58.130208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa25bc-0849-4e9b-8f60-4f71e0507da8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.152954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ae5ba-6f00-44d1-9bbd-99bfe042bba4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.200447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.201582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.202010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.202494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.222547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.224163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.224857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:58.225113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.225714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:58.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:58.235909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:12:58.236724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:58.239072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 170/341] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:12:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:12:58.240570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.248603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-94ddb3db-d141-4738-bf68-1dcdd41204ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:58.259815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:58.261536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 172/342] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:58.263033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526e-f773-409c-9ce6-2f97ccdecb37 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.264476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526e-f773-409c-9ce6-2f97ccdecb37 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.264476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526e-f773-409c-9ce6-2f97ccdecb37 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.264476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 173/343] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:58.265090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781526e-f773-409c-9ce6-2f97ccdecb37 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.364726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:58.387714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781526e-f773-409c-9ce6-2f97ccdecb37 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.422614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:58.428949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.429991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.430468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.430914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.434120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:58.434120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:58.437079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:58.451291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd7060-c735-49e7-b717-c728b2244e09 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.452038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 174/344] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:58.453096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.526119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.527368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.528123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.528977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.551795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7b6d1cbf-83e2-4f7d-b216-fb7ba6a7c9dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:58.570143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:58.571128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:12:58.571916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:12:58.576096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 171/345] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:58.578613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.579110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.579792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.602974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:12:58.604494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:12:58.618765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f4003-9f79-4eaf-88a5-d762511c644b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.662003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.663439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.663943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.664449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.665453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:58.665918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:58.667512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:58.679124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 175/346] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:58.680350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.683037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 172/347] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:12:58.684460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.764691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:58.764956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:58.812227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9e75-1830-43b9-b059-22fdd7a141eb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.823859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa901a-2603-4368-9e57-8eef9e264ba6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:58.864655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.866654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.867201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.867704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.868523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:58.868856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:58.869230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:58.869468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:58.869586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:58.869763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:58.871389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:58.882115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 173/348] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:58.888257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:58.955085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:58.956711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.009636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c740a-0bee-49cd-a5a1-cb5d31840b3a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.023119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.027472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.051924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.052935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.053417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.053857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.087064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 176/349] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:59.088196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.144185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.145933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.209044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc7e85-2f48-459f-bff9-47dcdaff43f4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.249860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.251070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.251507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.252976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.265833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:12:59.267332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 174/350] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:12:59.268454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:12:59.280243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:12:59.281297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:12:59.282093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:12:59.283542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.286132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 177/351] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:12:59.287550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.328358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.428449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99d352-e453-4756-87bd-8e6aff1b3fa0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.473282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11225-de9f-4d37-a1fc-50baf4e314c0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.477614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.478807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.479273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.479720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] PATH_INFO: `/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.481327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] RBAC: Authorizing `identity:get_user(user_id=a5362a3aa9b444ecb2e3a183a7573e0b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.483072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.484893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 175/352] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/users/a5362a3aa9b444ecb2e3a183a7573e0b => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:59.485718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.530618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.531585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.531871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.532150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.535332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.538846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.541757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 178/353] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:59.542641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.603697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.645359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c06ad-a5b4-4efc-8571-611f6d1e76eb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.676518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:12:59.696603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.698924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.699419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.699745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.702215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.702467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25bb0-8a50-4f52-a6ff-87758ef3e8a1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.705519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bfa27-a8e5-4e08-9844-027530bcc96b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.708844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 176/354] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:59.711117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638d18a-7a3d-43b7-895c-1af0dfd6e926 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.711611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638d18a-7a3d-43b7-895c-1af0dfd6e926 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.712093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638d18a-7a3d-43b7-895c-1af0dfd6e926 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.714822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 177/355] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:59.717098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae16e3-4fa1-4b42-a4c8-c7b5ce610644 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.718636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae16e3-4fa1-4b42-a4c8-c7b5ce610644 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.719103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae16e3-4fa1-4b42-a4c8-c7b5ce610644 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.721544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 178/356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:12:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:12:59.722928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae16e3-4fa1-4b42-a4c8-c7b5ce610644 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.746710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.747334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.747624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.747899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.748796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:59.749112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.750674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:12:59.760697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 179/357] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:12:59.761664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:12:59.915304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae16e3-4fa1-4b42-a4c8-c7b5ce610644 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.938843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6995c-5aa2-44ba-aabc-9b319bb52233 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:12:59.978787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:12:59.979851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:12:59.980375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:12:59.980810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:12:59.981949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] RBAC: Adding query filter params (role.id=1810bfbc8c644bfbbab1c652888b559f, user.id=a5362a3aa9b444ecb2e3a183a7573e0b, scope.system=all) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:12:59.982600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:12:59.984939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:00.006319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 179/358] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/role_assignments?user.id=a5362a3aa9b444ecb2e3a183a7573e0b&scope.system=all&role.id=1810bfbc8c644bfbbab1c652888b559f => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:00.007967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:00.008259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:00.009518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.009977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.010407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.016040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.021863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:00.024293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c355-098c-4dac-820a-4b519d82a3df None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:00.027470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 180/359] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:12:59 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:00.037176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045d86-619e-4dd5-a346-dd87f401b91e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.037603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045d86-619e-4dd5-a346-dd87f401b91e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.038043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045d86-619e-4dd5-a346-dd87f401b91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.065545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-16045d86-619e-4dd5-a346-dd87f401b91e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:00.066823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-16045d86-619e-4dd5-a346-dd87f401b91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:00.070891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.178541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 181/360] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:00.178541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.178541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.178541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.218419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:00.220021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:00.236026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b4956-abae-423f-8ed0-7d77d882f0e9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:00.300629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:00.301770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.302221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.302684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.306383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:00.309639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:00.314436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 180/361] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:00.316495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:00.348960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 182/362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:00.350600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:00.460388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.460988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.507400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757ca6-2451-4c75-9c57-ecfa7086ab3f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:00.521560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a174d37-bee8-4406-ab06-628b5dc6625d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:00.565319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:00.566360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.566859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.567347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.572597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:00.573278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.573563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.573842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.574705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=4a2d3aea849c46138a453442962777ab) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:00.575021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:00.576701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:00.583589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 183/363] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:00.584417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:00.725126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:00.727131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:00.776556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85fed9-2aea-42fd-9351-696c175ae82b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:00.825713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:00.826694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:00.827678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:00.828188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:00.831346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.853784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:00.863965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:00.875145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:00.877280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 184/364] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:00.879168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:00.915106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 181/365] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:00.916001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.086244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50124e3e-033a-438b-b33c-b91276ab4e79 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.097498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56538de3-8250-451e-aa47-0c520891e72d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.128465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.129562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.129944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.130226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.145393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:01.153829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:01.154968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:01.156551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 182/366] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:01.157630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.175021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.175983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.176625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.176860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.180853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=4a2d3aea849c46138a453442962777ab) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:01.180853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.183151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.196011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 185/367] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:00 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 260 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.197517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.209449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.294727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.324869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c774-9cd8-4725-972e-17e1be29c019 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.345802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4811-5631-4315-863b-2752e77f8a1f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.367391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.368094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.368578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.368858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.369659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:01.369965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.371459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.382522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 183/368] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.383541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.390483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.391394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.391680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.391959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.393381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.394921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.396997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 186/369] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.397710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.500021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.507997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb83703-e5b2-4616-86f4-3b8080d7d26c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.522431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03954-9bcf-4849-8886-d6aef40f2b52 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.547635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.548792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.549307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.549774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.550972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:01.551475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.553322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.553740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.563287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.564473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.564865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.565244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.565583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 184/370] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.566672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.643212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.644633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.687354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab295e72-ad7e-405c-8e30-2296b7c78727 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.726976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.728725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.729209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.729781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.731293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.733243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.735616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 185/371] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.736812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.759209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 187/372] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:01.760093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.766756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.791516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:01.853199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c518ed9-f412-43a2-a037-7e94fbea1344 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.889401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c0162-8248-4dd1-a35a-05554862d927 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:01.897419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.898426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.898561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.898920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.928047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:01.929229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:01.929777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:01.930214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:01.931219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:01.931699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.933435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:01.942338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 188/373] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:01.943879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:01.990125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:01.991741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.018824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:02.070729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd4bf-3076-4f81-9692-3c59a4e36a2f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.109303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 186/374] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:02.111935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.116351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.116351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.116351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.116841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:02.118687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:02.121213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.123172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.125693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 189/375] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:01 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:02.137530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:02.315750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbcd1b-ba75-4476-81af-e038d1281085 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.348487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:02.371884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.372811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.373100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.373454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] PATH_INFO: `/v3/projects/9702a09cdfa2459d8590360a884e14e2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.379573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] RBAC: Authorizing `identity:get_project(project_id=9702a09cdfa2459d8590360a884e14e2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.379573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.380584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 187/376] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:02 2022] GET /identity/v3/projects/9702a09cdfa2459d8590360a884e14e2 => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:02.390450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:02.394659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5140e5-f1e7-42eb-b26d-6ecda77611ec None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.474488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.476556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.477351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.478140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.481592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.484917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.489999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 190/377] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:02 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:02.500513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:02.531176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beae9e2-ab13-40d0-869f-967f9496a7d0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.564013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:02.591324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.592604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.592904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.593181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] PATH_INFO: `/v3/groups/e7c7a559b0b14bc794f7df9e498ad402` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.609229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] RBAC: Authorizing `identity:get_group(group_id=e7c7a559b0b14bc794f7df9e498ad402)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.610756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.612707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 188/378] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:02 2022] GET /identity/v3/groups/e7c7a559b0b14bc794f7df9e498ad402 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:02.622523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:02.691069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:02.707684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22479-6ba4-4b30-991a-ac710562df13 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.741888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a66c-50dd-4a2f-8ee6-843cfd40d36f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:02.779997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.780965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.781259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.781544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.782400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] RBAC: Adding query filter params (group.id=e7c7a559b0b14bc794f7df9e498ad402, role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=9702a09cdfa2459d8590360a884e14e2) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:02.782724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.784202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c288-065b-43f6-a2d9-07eea4cf75fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.789403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:02.790093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:02.790379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:02.790657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:02.791513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:02.791825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:02.793315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ee9cc-ec6f-4161-a355-34c5a1337eae None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:02.800096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 189/379] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:02 2022] GET /identity/v3/role_assignments?group.id=e7c7a559b0b14bc794f7df9e498ad402&scope.project.id=9702a09cdfa2459d8590360a884e14e2&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:02.805803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 191/380] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:02 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 260 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:03.659940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9808d-a9dd-42a3-9f72-06c8b8deacb7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:03.660687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9808d-a9dd-42a3-9f72-06c8b8deacb7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:03.661185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9808d-a9dd-42a3-9f72-06c8b8deacb7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:03.663170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 190/381] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:03.669534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:03.670160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:03.670891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:03.696992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:03.698552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:03.798593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 192/382] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:03.822884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9808d-a9dd-42a3-9f72-06c8b8deacb7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:04.002265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9808d-a9dd-42a3-9f72-06c8b8deacb7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:04.037235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:04.062525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:04.063785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.064276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.064770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.098105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:04.118722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:04.119674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:04.120472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:04.121545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 191/383] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:04.128709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:04.169128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:04.296678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c21f4-9a65-426e-a5dd-e8c5526adbd8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:04.367220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:04.368462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.368973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.369481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.370964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:04.371513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:04.374282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdb9f7-3c31-4dd9-8077-91e48c283802 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:04.390633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 193/384] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:04.396197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:04.558244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b316ab5-6613-4bfd-8fb0-5f3c914c8d0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:04.605468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:04.606365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.606650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.606924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.612091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74c0c0-b5bf-4de0-b8b7-62b1582feab8 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.613069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74c0c0-b5bf-4de0-b8b7-62b1582feab8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.613217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:04.613996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74c0c0-b5bf-4de0-b8b7-62b1582feab8 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.614702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:04.620363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 192/385] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:04 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:04.621840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 194/386] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:04.629759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:04.632101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.632420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.632704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.663655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:04.664935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:04.729723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:04.805826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 195/387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:04.844988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:04.858841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c68e1e-a905-4420-9a80-7f65cd15a25c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:04.886962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:04.958250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:04.960156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:04.960859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:04.961424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:04.962970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:04.965038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:04.967927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 193/388] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:04 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:04.981651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:05.079993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fa8cb-af6f-4948-9515-db25fd790251 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:05.150516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:05.151822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.152400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.152858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.190701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a0953c6c-2ec3-4b9f-9d5c-3344649e7137 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:05.203738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:05.205540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:05.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 196/389] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:05.207908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b814-0ed1-4e41-8918-87364c3eefe4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.207908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b814-0ed1-4e41-8918-87364c3eefe4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.209019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b814-0ed1-4e41-8918-87364c3eefe4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.212507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 197/390] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:05.220079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.220517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.220980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.224292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56d5ae-a620-4723-8b46-634ff12e2378 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:05.246604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:05.264759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:05.267918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:05.279783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:05.281378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.281909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.282613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.286674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, group_id=c320b8c9b0164683a2b6ccc9d3f2a95e, role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:05.289446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:05.324368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 194/391] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jun 27 15:13:04 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:05.324368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:05.486220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 198/392] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:05.530990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b1d80-74fe-4e4b-a2b3-ce58fa782655 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:05.532458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:05.535067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:05.584839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:05.585550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.585835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.586103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.586885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:05.587190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:05.588649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:05.602222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 195/393] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:13:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:05.609366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:05.682553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e224-50b5-4036-9872-d4cb2d4e6ea9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:05.687524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:05.732682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:05.734649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.735460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.736192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.758113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:05.774697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:05.776533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:05.778072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:05.784471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 199/394] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:05.804753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:05.875716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661a1bb-a4a9-47ea-bc20-c80ef5b8cca7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:05.927659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:05.946258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:05.948165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.948967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.949747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.952288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:05.955333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfeec20-3ff6-4682-a3a7-30205728596c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:05.958834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 196/395] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:05 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:05.963511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593345-75f6-4fd5-b6f0-4c318c018820 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.964474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593345-75f6-4fd5-b6f0-4c318c018820 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.965095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593345-75f6-4fd5-b6f0-4c318c018820 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.968699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 197/396] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:05.971638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28020221-e7d6-4ca4-9fe7-fe40553481a2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:05.972352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28020221-e7d6-4ca4-9fe7-fe40553481a2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:05.973016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28020221-e7d6-4ca4-9fe7-fe40553481a2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:05.976485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 198/397] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:05.978445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28020221-e7d6-4ca4-9fe7-fe40553481a2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.001969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea2017-df71-4468-91d5-96ca9c75c55c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.058210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.058809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.059090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.059364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.060314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:06.060787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:06.062526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d87b1-d94e-40f9-901f-454e79cbd360 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:06.074890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 200/398] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:06.076755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.077187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.077595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.104651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:06.106618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:06.179813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28020221-e7d6-4ca4-9fe7-fe40553481a2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.190095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:06.224763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.229799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.229799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.229799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.231450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:06.233704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b787a0-c6c3-402d-8113-d5ecd4a01a9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:06.235422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 199/399] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:05 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:06.236492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.236803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.239894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.257993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 201/400] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:06.261701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.267459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:06.269569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:06.348178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:06.380717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 200/401] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:06.381922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.421916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa2a2b-468e-4eaa-9c58-c3fb6401bdeb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.471288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.472931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.473433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.473928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.476088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:06.478934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:06.482180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 202/402] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:06 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:06.489371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.532669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded63c4-ea69-483a-a7be-6bb038e4d504 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.536749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:06.581966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.583496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.583924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.584359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.587757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2d3aea849c46138a453442962777ab, group_id=c320b8c9b0164683a2b6ccc9d3f2a95e, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:06.590290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:06.620478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 201/403] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jun 27 15:13:06 2022] PUT /identity/v3/projects/4a2d3aea849c46138a453442962777ab/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:06.620478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.673135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b7d0b-2d5c-4171-b8e0-92980e3c1b5d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.704823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:06.722274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.723132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.723415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.723734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.738588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:06.746916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:06.748740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 203/404] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:06.750307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.882918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dd824-e4cd-4562-bd03-8f8811e3d383 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.884744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bd0e4-ab27-47df-afef-3cb6b1155982 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:06.934730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.937192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.937192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.937419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.939550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:06.942930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:06.947020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 204/405] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:06 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:06.948484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:06.948990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:06.950021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:06.950474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:06.950913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:06.970044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:06.986000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:06.986960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:06.987746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:06.988690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 202/406] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:06.990247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.011587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:07.035160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:07.122844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec621f-8b1c-466d-b387-064016b94ae8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:07.179241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd15a7-d291-417a-b3fd-2f527031bfba None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:07.201007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:07.202732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:07.203674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:07.204476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:07.206197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:07.206969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:07.210249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:07.236108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 205/407] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:07.238362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.250153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:07.251701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:07.252176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:07.252685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:07.259288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, group_id=c320b8c9b0164683a2b6ccc9d3f2a95e, role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:07.262752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:07.291331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 203/408] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jun 27 15:13:06 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:07.293371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.533325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fcad9-b137-469b-a578-d669d40917df None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:07.544863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:07.563236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:07.619627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:07.620922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:07.621405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:07.621877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:07.622581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff24342-7ce6-4c48-91a3-1e1328d5a78e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:07.623308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:07.623950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:07.627462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:07.646661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 206/409] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:13:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:07.654662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.693729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:07.695229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:07.695729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:07.696216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] PATH_INFO: `/v3/groups/e7c7a559b0b14bc794f7df9e498ad402` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:07.699592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] RBAC: Authorizing `identity:get_group(group_id=e7c7a559b0b14bc794f7df9e498ad402)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:07.703077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:07.707365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 204/410] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:07 2022] GET /identity/v3/groups/e7c7a559b0b14bc794f7df9e498ad402 => generated 229 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:07.715953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.787123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc5607-ca4f-40d7-a50d-46fb98a78c66 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:07.856353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:07.856353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:07.856353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:07.856353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:07.872381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:07.872381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:07.873782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 207/411] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:07 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:07.882604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:07.949027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:07.953293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06550-2bb9-4a8f-972a-08791279c401 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:08.014916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe24f1-8f4a-4be9-9dee-bde19587a4f0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:08.033570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:08.035260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.035766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.036269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] PATH_INFO: `/v3/projects/9702a09cdfa2459d8590360a884e14e2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.041881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] RBAC: Authorizing `identity:get_project(project_id=9702a09cdfa2459d8590360a884e14e2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:08.044548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:08.048590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 205/412] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:07 2022] GET /identity/v3/projects/9702a09cdfa2459d8590360a884e14e2 => generated 344 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:08.062175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:08.063129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:08.064952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.065729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.066459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.069550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:08.072573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:08.076606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 208/413] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:07 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:08.084785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:08.144593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:08.329129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8384edd-eef4-4d9e-b052-56edd8fd61d7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:08.330292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659243-6d06-4fb9-bd3b-1ece5b479281 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:08.351748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:08.399104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:08.401192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.401585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:08.402300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.403049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] PATH_INFO: `/v3/projects/9702a09cdfa2459d8590360a884e14e2/groups/e7c7a559b0b14bc794f7df9e498ad402/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.403203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.403624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.404104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.407805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9702a09cdfa2459d8590360a884e14e2, group_id=e7c7a559b0b14bc794f7df9e498ad402, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:08.408125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] RBAC: Authorizing `identity:create_grant(project_id=28dfe7f807c94a068f1e3d10c27a4d33, group_id=c320b8c9b0164683a2b6ccc9d3f2a95e, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:08.410804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc28f09-1179-417a-a48b-260209c533dd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:08.411599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715d94e-23e4-4350-9589-eddb7855afb5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:08.438473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 206/415] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jun 27 15:13:08 2022] PUT /identity/v3/projects/9702a09cdfa2459d8590360a884e14e2/groups/e7c7a559b0b14bc794f7df9e498ad402/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:08.439902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 209/415] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jun 27 15:13:08 2022] PUT /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33/groups/c320b8c9b0164683a2b6ccc9d3f2a95e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:13:08.441854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612db28-c475-432f-9f0e-6b01982e4472 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.442800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612db28-c475-432f-9f0e-6b01982e4472 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.444009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612db28-c475-432f-9f0e-6b01982e4472 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.447819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 207/416] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:08.462444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:08.462901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:08.463248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:08.509792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:08.511885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:08.684294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 210/417] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:08.762923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612db28-c475-432f-9f0e-6b01982e4472 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:08.929503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:08.968202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612db28-c475-432f-9f0e-6b01982e4472 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:09.036439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:09.037913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:09.038346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:09.038768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:09.059543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:09.074068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:09.075028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:09.075819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:09.080606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 208/418] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:09.085186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:09.258130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:09.262577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ce0fa-7d5d-4ae3-a0bd-ca75c31d9ba7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:09.310005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:09.310714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:09.311153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:09.312189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:09.312405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:09.312714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:09.314124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a431a2d-0061-450a-bf7e-11507f283594 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:09.324602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 211/419] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:09.334223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:09.558468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab4d4d-07dd-4055-96dc-7bf030177b18 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:09.626003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:09.627455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:09.628124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:09.628815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:09.650305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:09.760947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:09.763914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:09.947533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045772-5f8b-482c-9255-66bf0472164c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:09.948110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045772-5f8b-482c-9255-66bf0472164c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:09.948634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60045772-5f8b-482c-9255-66bf0472164c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:09.951543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 212/420] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:09.960212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:09.960705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:09.961181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:09.969173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 209/421] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:09.981486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:10.018300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:10.020467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:10.150445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:10.242417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 213/422] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:10.279710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254458de-b795-482c-96e9-cf4958e505a2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:10.308919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:10.328107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:10.329053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:10.329424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:10.329711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:10.337345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:10.338943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:10.341112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 210/423] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:09 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:10.352365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:10.489156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:10.511102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b46a3-4926-4fd6-9acf-45aea6f4d32f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:10.595954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d459c3-3087-4b93-a044-4ebbd6843c59 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:10.618540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:10.649422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:10.650675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:10.651112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:10.651544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:10.680052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:10.689746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:10.691359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:10.691884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:10.692441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:10.696014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:10.697262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:10.698042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:10.699230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 214/424] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:10.700536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:10.703455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:10.705639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:10.724039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 211/425] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:10 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:10.738826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:10.950326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2a23e-8458-4df1-b58d-89d3720d0625 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:11.012281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40178f55-3fa5-44d6-965a-984fa315c0de None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:11.026954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:11.028211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.028754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.029303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.030842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:11.031397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:11.034189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe84dc4-4b95-4270-9158-1b9046353baf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:11.060455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 215/426] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:13:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:11.063078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5cc6-3c3d-43ac-9cef-7042d6dea370 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.063591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5cc6-3c3d-43ac-9cef-7042d6dea370 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.064089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f5cc6-3c3d-43ac-9cef-7042d6dea370 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.067212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 216/427] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:11.069060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5cc6-3c3d-43ac-9cef-7042d6dea370 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:11.083610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:11.084777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.085263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.085767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.087415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=4a2d3aea849c46138a453442962777ab) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:11.087992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:11.090902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee1d608-73d0-45b7-b85d-25500a1d28f9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:11.103545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 212/428] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:10 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 611 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:11.105917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.106439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.106947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.108284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:11.137681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:11.139953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:11.221749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:11.358842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 213/429] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:11.385989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f5cc6-3c3d-43ac-9cef-7042d6dea370 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:11.406386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:11.464505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:11.466675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.467703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.468871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.606880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:11.608873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbc02a-97a3-4a0f-9fde-42986aa3daf1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:11.626497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510eca4-d261-4372-943f-1947d14eb978 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:11.653577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:11.690048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:11.691516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.692141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.692815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.713808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:11.718940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 217/430] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:11.720054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7a576-3bd3-48c2-8928-3c935edbd91b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.720365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7a576-3bd3-48c2-8928-3c935edbd91b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.720992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7a576-3bd3-48c2-8928-3c935edbd91b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.722317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 218/431] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:11.723397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357391c3-0deb-4083-afac-ecbb7f3d6448 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.723678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357391c3-0deb-4083-afac-ecbb7f3d6448 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.723953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357391c3-0deb-4083-afac-ecbb7f3d6448 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.725467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 219/432] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:11.728145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357391c3-0deb-4083-afac-ecbb7f3d6448 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4073ce46-f5fa-46e3-8ce9-6c7285ef4408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:11.731995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:11.733754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:11.734742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 214/433] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:11.737227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfbc1f-adf2-4b02-948c-4d29fd60a17c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.737627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfbc1f-adf2-4b02-948c-4d29fd60a17c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.738098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfbc1f-adf2-4b02-948c-4d29fd60a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.766018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-03cfbc1f-adf2-4b02-948c-4d29fd60a17c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:11.768062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-03cfbc1f-adf2-4b02-948c-4d29fd60a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:11.806435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:11.853389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357391c3-0deb-4083-afac-ecbb7f3d6448 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:11.869557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 215/434] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:11.871098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.871462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.871931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.874186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:11.894469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:11.896181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:11.901615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702718-8683-490a-b828-8ec815a64911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:11.902662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702718-8683-490a-b828-8ec815a64911 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:11.903033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702718-8683-490a-b828-8ec815a64911 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:11.903372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702718-8683-490a-b828-8ec815a64911 None admin] PATH_INFO: `/v3/projects/4a2d3aea849c46138a453442962777ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:11.905708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702718-8683-490a-b828-8ec815a64911 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2d3aea849c46138a453442962777ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:11.907413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702718-8683-490a-b828-8ec815a64911 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:11.909629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 220/435] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:11 2022] GET /identity/v3/projects/4a2d3aea849c46138a453442962777ab => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:11.910484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702718-8683-490a-b828-8ec815a64911 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:11.994176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 216/436] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:11.995583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.050440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702718-8683-490a-b828-8ec815a64911 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.090765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.091433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.091716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.092156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.093215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:12.093611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.095131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.106039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 221/437] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:12.107406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.121385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.136193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaa292-334e-4b59-9b43-9628bb3580e8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.166683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.190950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.191842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.192179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.192541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.205629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:12.213884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:12.215134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 217/438] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:12.216640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.240558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b332e-1dca-4033-ac65-939726785ea2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.276621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.277502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.277785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.278063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.279279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.280916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.282592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 222/439] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:12.283376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.378682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5772cb-0527-4845-a020-15c8cca0aeea None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.405491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d4e49-2fc3-476c-94dc-88afe22b47ac None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.428024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.442452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.444217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.444774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.445286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.458591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.459306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.459626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.460006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.465009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:12.477077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:12.478846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:12.480419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:12.482731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 218/440] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:12.484199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.506566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.561848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.563551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.619091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5cdbf-78c5-403c-9d77-78505849e847 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.673865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.674660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.675088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.675410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.676445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:12.676800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.678498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.689325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 219/441] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:12.690720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.710601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 223/442] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:12.711482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.732389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.787361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:12.829859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c4ddf-ad91-4f02-990b-993d0841f315 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.868915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c4d8-db17-472e-be3a-c38eb417bd3a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:12.886421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.887457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.887898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.888304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.889382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=4a2d3aea849c46138a453442962777ab) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:12.889757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.891327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.898157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 220/443] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=4a2d3aea849c46138a453442962777ab&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:12.899050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:12.932243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:12.933389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:12.933744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:12.934138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:12.935252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:12.935756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:12.937960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:12.957296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 224/444] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:12.957296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.051304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102930f6-498d-451d-a2f7-21e22a672493 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.063788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.099041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.099612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.100983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.100983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.111419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.112174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7f695-4b1f-429d-91d2-698088998607 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.173597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.175166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.176113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.176240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.178580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.181380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.184806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 225/445] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:13.186380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.208894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.210419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.358315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719881e-1e88-471d-b900-0952de5c0a32 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.383060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 221/446] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:13.384939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.414747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.415522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.415703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.415992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.430245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.448446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.530647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.532733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.590927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831b56-f22c-4978-826a-feb1ff077a0b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.630187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.631142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.631570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.631931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.634184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.635717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.637538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 222/447] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:13.638379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.714771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 226/448] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:13.715582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.765111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.796355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfbbaf-5f2c-4180-935d-1aae1aad1647 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.800119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:13.840170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2a4e-9770-496c-a745-f4ccc139a69a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:13.847881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.849256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.849673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.850098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] PATH_INFO: `/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.852097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] RBAC: Authorizing `identity:get_project(project_id=28dfe7f807c94a068f1e3d10c27a4d33)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.854550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.857714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 223/449] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/projects/28dfe7f807c94a068f1e3d10c27a4d33 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:13.858972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:13.891571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:13.892235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:13.892626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:13.892963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:13.893895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:13.894266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:13.896177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:13.904268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 227/450] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:13.905907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:14.045159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6e436-e19d-4165-8ceb-077837f759e7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:14.075334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ef9f8-ab8e-4660-830c-847bc0c9bb78 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:14.089549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:14.118348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:14.119745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:14.120188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:14.120679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] PATH_INFO: `/v3/projects/9702a09cdfa2459d8590360a884e14e2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:14.122495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] RBAC: Authorizing `identity:get_project(project_id=9702a09cdfa2459d8590360a884e14e2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:14.125423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:14.128935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 224/451] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/projects/9702a09cdfa2459d8590360a884e14e2 => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:14.135055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:14.138044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:14.139094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:14.139450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:14.139910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] PATH_INFO: `/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:14.142284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] RBAC: Authorizing `identity:get_group(group_id=c320b8c9b0164683a2b6ccc9d3f2a95e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:14.143686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:14.145435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 228/452] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:13 2022] GET /identity/v3/groups/c320b8c9b0164683a2b6ccc9d3f2a95e => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:14.151805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:14.200503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:14.269511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1c66-0ac0-4ce8-aa46-d65a45f8a113 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:14.282572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6bbb8-aae1-48d5-880d-ba2d79f19d1d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:14.307704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:14.308764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:14.309163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:14.309577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] PATH_INFO: `/v3/groups/e7c7a559b0b14bc794f7df9e498ad402` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:14.311293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] RBAC: Authorizing `identity:get_group(group_id=e7c7a559b0b14bc794f7df9e498ad402)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:14.313329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:14.315444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 225/453] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jun 27 15:13:14 2022] GET /identity/v3/groups/e7c7a559b0b14bc794f7df9e498ad402 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:14.321907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:14.322388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:14.322584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:14.322854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:14.323130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:14.323981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] RBAC: Adding query filter params (group.id=c320b8c9b0164683a2b6ccc9d3f2a95e, role.id=0a2c65ac0f7c41fd8d3b88920a4f73de, scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:14.324291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:14.325834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f25cc-1e27-47a2-bedd-287d5f896308 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:14.332455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 229/454] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:14 2022] GET /identity/v3/role_assignments?group.id=c320b8c9b0164683a2b6ccc9d3f2a95e&scope.project.id=28dfe7f807c94a068f1e3d10c27a4d33&role.id=0a2c65ac0f7c41fd8d3b88920a4f73de => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:14.443159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d41af6-c1a8-4469-ab5a-dc418f5b9ea1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:14.455495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:14.487033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:14.487791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:14.488134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:14.488516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:14.489474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] RBAC: Adding query filter params (group.id=e7c7a559b0b14bc794f7df9e498ad402, role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=9702a09cdfa2459d8590360a884e14e2) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:14.489849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:14.491432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f9090-b5b2-4680-a4f0-1b1c13df701f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:14.499590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 226/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jun 27 15:13:14 2022] GET /identity/v3/role_assignments?group.id=e7c7a559b0b14bc794f7df9e498ad402&scope.project.id=9702a09cdfa2459d8590360a884e14e2&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:17.742460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17671c59-7ca9-4b2c-9fa7-48c04167f588 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:17.743052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17671c59-7ca9-4b2c-9fa7-48c04167f588 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:17.743052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17671c59-7ca9-4b2c-9fa7-48c04167f588 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:17.744716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 230/456] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:17.750219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190c3a-028d-4f1b-b2fc-7b86922d6a6d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:17.751138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190c3a-028d-4f1b-b2fc-7b86922d6a6d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:17.751632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190c3a-028d-4f1b-b2fc-7b86922d6a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:17.781512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-af190c3a-028d-4f1b-b2fc-7b86922d6a6d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:17.783117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-af190c3a-028d-4f1b-b2fc-7b86922d6a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:17.881822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 227/457] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:17.894426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864b44-987c-4ed0-919a-b7f2774c58a1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:17.894757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864b44-987c-4ed0-919a-b7f2774c58a1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:17.894997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0864b44-987c-4ed0-919a-b7f2774c58a1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:17.896411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 231/458] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:17.902006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f315e-b24f-4b6a-8f5e-70f69c8ae54d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:17.902291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f315e-b24f-4b6a-8f5e-70f69c8ae54d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:17.902574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f315e-b24f-4b6a-8f5e-70f69c8ae54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:17.921961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-166f315e-b24f-4b6a-8f5e-70f69c8ae54d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:17.923216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-166f315e-b24f-4b6a-8f5e-70f69c8ae54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:17.956619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0864b44-987c-4ed0-919a-b7f2774c58a1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.020451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 228/459] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:18.024785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df60e8c-6012-4e25-b9e6-a69357cff154 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.025184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df60e8c-6012-4e25-b9e6-a69357cff154 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.025498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df60e8c-6012-4e25-b9e6-a69357cff154 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.027213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 229/460] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:13:18.036308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.036636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.037089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.052629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.062667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:18.064712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:18.116774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.124093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0864b44-987c-4ed0-919a-b7f2774c58a1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.164991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.165935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.166230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.166472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.177561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.179057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.189375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 230/461] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:18.190687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-df8329e1-af8f-4155-bb78-bddbb7346e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:18.203040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:18.203960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:18.205000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:18.205000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:18.205000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.205000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:18.212635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 232/462] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:18.214467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1b7b-7143-449a-aa05-82d813a743ad None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.214765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1b7b-7143-449a-aa05-82d813a743ad None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.215391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d1b7b-7143-449a-aa05-82d813a743ad None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.218199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 233/463] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:18.218199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d1b7b-7143-449a-aa05-82d813a743ad None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.320238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.333037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d1b7b-7143-449a-aa05-82d813a743ad None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.349037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7aadb-6121-4dc1-874c-370adb6f791d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.373953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.374763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.375054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.375974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.387401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.388977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.392937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.393759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.394041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.394319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.406292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.407785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:18.409766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:18.411280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 234/464] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:18.412973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d22aae99-8d77-48bb-a5f7-f62e24ec7a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:18.434426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:18.436093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 231/465] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.437808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.459036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:18.460253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:18.472703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.537761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596c3e2-e6d4-4dcd-9022-61f02aeffad9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.571923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 232/466] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:18.573183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.594890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.596131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.596727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.597286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.598708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:18.599788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.602240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.623234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 235/467] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:18.628258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.629124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.692304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a3af-7168-4266-96fc-11c91138e0f0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.734845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.736077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.736805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.737469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.738929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:18.739494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.741948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.774087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95744b-50a5-4e08-a762-ff7e6538a30a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.780147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 233/468] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:18.782299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.790083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.823305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.823885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.824169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.824457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.825220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:18.825500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.826953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:18.839051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 236/469] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:18.839865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:18.910737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:18.914842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ed9bf-ac40-49fe-a740-e7773e134a3a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.962029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:18.963307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:18.963758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:18.964183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:18.964279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553385e-5edc-47cd-be8e-871518e7e589 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:18.981043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:18.983460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:19.006887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:19.007833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:19.008754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:19.008754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:19.008754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:19.008754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:19.008754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 234/470] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:19.009722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.012630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.013326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.013617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.013897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.014766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.016540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c5fec-0902-43ec-abed-08a27c09f4a9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.040495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 237/471] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:18 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:19.042542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2613e9-b008-41a3-8572-da71bdfd38c8 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.042985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2613e9-b008-41a3-8572-da71bdfd38c8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.043490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2613e9-b008-41a3-8572-da71bdfd38c8 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.051644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 238/472] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:19.053098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2613e9-b008-41a3-8572-da71bdfd38c8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.090452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:19.151379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c376a5-f6bc-417d-ae26-42891dd0fa21 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:19.197494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2613e9-b008-41a3-8572-da71bdfd38c8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:19.210691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.211406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.211688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.211964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.212727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.214219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.222944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:19.240548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 235/473] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:19 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:19.242605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.245405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.246148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.246454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.246712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.247449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.249396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee5b6-05ae-4b1e-947f-65d0bca40653 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.284847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 239/474] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:19 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:19.288085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.288704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.289349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.332009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:19.335637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:19.446470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764e814-63a2-4b46-8316-a3047bdede12 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:19.482254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 240/475] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:19.482481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:19.506599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.524294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.525018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.525308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.525596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.526344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:19.526660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.528115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.547976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 236/476] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:19.549424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.665949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38c2c1-2560-43be-8120-deed08d8f12d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:19.669538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:19.696706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7fff5-051d-4afb-b819-ab63f15953b6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:19.709125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.709826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.710103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.710373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.711177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.712703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.719760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.sql.core [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:13:19.719760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 15:13:19.719760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [parameters: {'id': '7baf0adfa94d4e4589a3fc26018ae09f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 15:13:19.719760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Jun 27 15:13:19.738147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:13:19.739072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application yield resource Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise self.value Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application session.commit() Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:13:19.739986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:13:19.741400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:13:19.743243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:13:19.745449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [parameters: {'id': '7baf0adfa94d4e4589a3fc26018ae09f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:19.747384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 15:13:19.749214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:19.757767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 241/477] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jun 27 15:13:19 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:19.757767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:19.760826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:19.762140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:19.762682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:19.763127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:19.764186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:19.767888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:19.824561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 237/478] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:19 2022] POST /identity/v3/users => generated 249 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:13:19.989079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13bbf-9c05-4ade-96aa-9cea8d7e55e6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:20.046039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:20.046913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:20.047200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:20.047476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:20.060036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a4232357-14cd-40ea-bc34-8ce850a6b760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:20.072748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:20.073737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:20.074528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 242/479] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:13:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:20.080616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:20.127308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:20.229272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aefa67-cfbe-493d-877c-c367328361fd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:20.283138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:20.283841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:20.284125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:20.284441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:20.285214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:20.285524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:20.286948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc45f89-dbe3-4a7f-a20b-2f6b854438df None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:20.301202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 238/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:21.744293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253897d-3550-4558-854f-d3f939b733c2 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:21.745390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253897d-3550-4558-854f-d3f939b733c2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:21.745955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253897d-3550-4558-854f-d3f939b733c2 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:21.749393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 243/481] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:21.755624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb535-aa2c-4371-9db7-aa57d571660f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:21.756620 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb535-aa2c-4371-9db7-aa57d571660f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:21.757110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bb535-aa2c-4371-9db7-aa57d571660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:21.780797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-787bb535-aa2c-4371-9db7-aa57d571660f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:21.782434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-787bb535-aa2c-4371-9db7-aa57d571660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:21.894099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 239/482] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:21.930197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253897d-3550-4558-854f-d3f939b733c2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:21.970249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.028406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50df1-f7b8-4ff3-9b57-1a02edc0d488 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.029081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50df1-f7b8-4ff3-9b57-1a02edc0d488 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.029634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50df1-f7b8-4ff3-9b57-1a02edc0d488 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.031539 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 240/483] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:22.036902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.037579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.038069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.063318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253897d-3550-4558-854f-d3f939b733c2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.075541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:22.077007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:22.107424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.130067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.130898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.131180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.131456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.156715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c13c5960-f034-4141-ad38-e0a44d7d2ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:22.162222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:22.163154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.163942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:22.164599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 244/484] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:22.168175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654bd18-5ca6-4262-a77d-9ca21e7ce914 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.168175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654bd18-5ca6-4262-a77d-9ca21e7ce914 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.168175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654bd18-5ca6-4262-a77d-9ca21e7ce914 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.168175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 245/485] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:22.176804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.177084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.177370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.192166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 241/486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:22.200355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.204203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:22.205525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:22.303912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 246/487] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:22.304826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.326083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.354199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec030-e33d-4fa0-bb15-3177107485c3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.398908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.400264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.400778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.401344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.402569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:22.403045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:22.404687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:22.416904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 242/488] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:22.418010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c76d-80b6-425c-911f-b78cccc4431b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.418122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.437185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.458218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.459188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.459542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.459969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.472159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:22.480641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:22.481562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.482359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:22.487327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 247/489] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:22.488530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.559295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5333292-6192-4c36-85a0-3a70420398fb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.606666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.607555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.607879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.608224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.609441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.623459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:22.632065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:22.633239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:22.634014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 243/490] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:22.634970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.649218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27534fb3-072d-4d4a-b768-e57ad43c5e49 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.697242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.698348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.698768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.699175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.715696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:22.718498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:22.722664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:22.741881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:22.742798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:22.743630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:22.743630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:22.743630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:22.743630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:22.744666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 248/491] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:22.746745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.766408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ef9cc-f435-4467-8d83-dff67acda5a0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.808563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.809492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.809786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.810031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.811095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:22.811410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:22.813208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:22.828633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 244/492] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:22.829732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.890171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a49d-0ce2-4102-944a-8c4202a1a9b9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:22.938138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:22.946595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:22.948022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:22.948771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:22.949620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:22.952377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:22.953174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:22.954837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:22.966046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 249/493] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:22.967576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:22.995918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f64a9-9d21-4d66-b227-c02b7ef99666 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.039574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.040262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.040591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.040870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.041615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:23.041926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.043454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.045287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.060729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 245/494] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:23.062029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.103565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3799e7-6b33-4f78-9c05-b1207501d769 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.148235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.150133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.150644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.151402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.166746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.168651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:23.191207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:23.192123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:23.194236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:23.194236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:23.194236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:23.194236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:23.194838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 250/495] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:23.196144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.225970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82200c-76bc-4bf7-9e9c-6c23437eb0af None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.263758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.282780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.283946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.284436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.284757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.302239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.304873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.326287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904d206-5dc5-466c-a65a-3f0ea39a16c8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:23.336879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:23.337784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:23.338609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:23.338609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:23.338609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:23.338609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:23.338811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 246/496] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:23.339805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.375815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.376885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.377263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.377717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.389875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.391391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.397783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-363d6811-e0c6-4230-8555-ce3939de935d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:23.408458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:23.409631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:23.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 251/497] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:23.413705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582b22-56f4-4823-bac3-21df4ebedb82 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.413959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582b22-56f4-4823-bac3-21df4ebedb82 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.415158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582b22-56f4-4823-bac3-21df4ebedb82 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.416835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 252/498] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:23.417580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582b22-56f4-4823-bac3-21df4ebedb82 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.471674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a3db4-32cb-4a15-8b64-98e693b8839d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.512254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.512963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.513235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.513507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.514366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:23.514666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.516076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.529020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 247/499] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:23.529806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.537734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582b22-56f4-4823-bac3-21df4ebedb82 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.541269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.577362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.577847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.578135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.578415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.579161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:23.579472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.580944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8307752-f431-42fc-b088-cbf7641a0965 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.593821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 253/500] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:23.595250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.595540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.595932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.616570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:23.617841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:23.646091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8254a64-7522-451c-bf74-c577b21cc9f1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.655078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.689072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.689766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.690073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.690324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.691193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:23.691502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.692978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c55dff-122c-4af0-87b3-604453172f4e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.694083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 254/501] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:23.694951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.740570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 248/502] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:23.742864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b85cc4-1f1c-4e0a-a4f8-53021595c7c5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.743265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b85cc4-1f1c-4e0a-a4f8-53021595c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.743617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b85cc4-1f1c-4e0a-a4f8-53021595c7c5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.745057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 249/503] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:23.746181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b85cc4-1f1c-4e0a-a4f8-53021595c7c5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.824227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:23.832864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a7efc-f148-4a8c-8a4c-9f961e8ffead None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.871290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.872229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.872579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.872822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.879775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b85cc4-1f1c-4e0a-a4f8-53021595c7c5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:23.881796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.884229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:23.906047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:23.907947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 255/504] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:23.909602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:23.923588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:23.924438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:23.924719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:23.924989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:23.939420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:23.940950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:23.959612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:23.961154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 250/505] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:23.962244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.002210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.050197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d4a1c-c42a-442d-bd3d-a35a5ec4e356 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.088060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.088821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.089127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.089408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.100687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.102266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.103813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807ca3-1b7f-4f60-862d-78b052b2c136 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-910e3734-76fd-417e-ac73-8fd82105d902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:24.126757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:24.127882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:24.129548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:24.129548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:24.129548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:24.129548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:24.129548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 256/506] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:24.130358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.130758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.131117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.143960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.148592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.149254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.149568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.149925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.152199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:24.153721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:24.228036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.229550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 257/507] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:24.229733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.230626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.260358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.355105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e240a-dd6d-4db6-8e7b-404f766d5cae None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.384904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 251/508] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:24.385474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.393507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.394202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.394487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.394766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.395657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:24.395923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.397456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.409549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.444178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 258/509] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:24.444178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.509674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39ea53-7f07-44ba-bbee-2a5a2cba9d06 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.535049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.548764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.549722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.549995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.550836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:24.551138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.552645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.557261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164903d-c4e8-4a56-a50a-d102fb4e51b7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.581005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 252/510] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:24.581005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.614499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.615460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.615884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.616213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.617501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:24.617985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.620350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.645067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 259/511] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:24.645067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.724392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.741568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b623e-fae5-4bd4-be3e-8c93f8d9f522 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.783371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e52d-7b9d-4ee5-b6d2-7d8ada1f1c20 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:24.784480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.785320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.785615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.786833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.803638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:24.820728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:24.821862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:24.822825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:24.829867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 253/512] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:24.829867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.830797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:24.831605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:24.831912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:24.832232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:24.848828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:24.850366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:24.872306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:24.874232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 260/513] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:24.876023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:24.913698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:24.990581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bc1d8-3182-4b6e-8aaa-819d5c55d87d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.012594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f4cf4-bbca-4434-95b2-cee60f0cf760 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.021533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.044042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.044745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.045025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.045309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.064543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.065357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.065724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.066008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.135399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.137773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.153752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.156150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.200172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.277079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 254/514] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:25.278190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.306465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 261/515] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:25.307347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.320147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.407449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbdab2-e219-494e-aa37-8f889893cfe5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.453956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.455406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.455997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.456405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.457218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.458784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.459067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dca622-732f-4801-b9cd-52552a8e44c8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.478780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.487665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 255/516] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:25 2022] POST /identity/v3/users => generated 247 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:25.488978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.534189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.535107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.535567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.536176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.537888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:25.538441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.541022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.556266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 262/517] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:25.557901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.668971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.675759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e85bfc-af06-46d9-95f4-eb719009cdd1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.729555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.730472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a375b4-6ad0-43db-a722-8cc26338c541 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.730681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.731079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.731527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.733653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:25.735401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.737488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.758783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 256/518] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:25.759688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.787995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.789227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.789684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.790142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:25.805067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:25.806917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:25.830428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:25.832062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:25.833493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:25.833493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:25.833493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:25.833493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:25.836554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 263/519] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:25.837195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:25.880752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:25.941493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f03cf-e6cf-4b7b-ae25-66029f8cf562 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:25.989308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:25.990383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:25.990936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:25.991384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.002882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50746ff-3b40-4f54-9ad4-21ce6baa4ea1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.006991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.006991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:26.028432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:26.030166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 257/520] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:26.032921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.039563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.062915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.064707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.064707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.065206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.080482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.082239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:26.109916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:26.111586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 264/521] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:26.112494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.194934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccd918-196f-4c67-8d13-bad64baa3465 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.210137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.243000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.243813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.244094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.244401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.252798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.254465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:26.271461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:26.273176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 258/522] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:26.274830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.285492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0ca44-36b2-41cf-ac12-0abfd2ca332a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.334875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.335747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.336147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.336580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.337478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:26.337944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.339718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.355704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 265/523] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:26.356478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.399112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.435010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aeb23-62ab-4c7f-84a2-257bbd526801 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.477481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.478516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.479020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.479373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.480510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:26.481064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.484627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.503489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 259/524] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:26.504388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b847b-30b8-47be-b7bd-025aa2f865b8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.505020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.553196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.553976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.554260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.554535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.555253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:26.555673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.557258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.562023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.580370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 266/525] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:26.580370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.664632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e929c-55d6-46c2-9cba-b7b8847f3560 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.711141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.711901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.712348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.712779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.714116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:26.714830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.717562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.727574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.751119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ed47e-9c6a-4b45-bbf8-b85a9c6506c9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.753326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 260/526] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:26.757219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.801067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.802535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.803181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.803534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.818687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.821513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:26.839551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:26.846954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 267/527] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:26.848064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:26.896021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:26.912574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6933ae-508b-4bcb-8662-925913e6fc8a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:26.960301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:26.961585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:26.961996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:26.962393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:26.971870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:26.976935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:26.997300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:26.998252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:26.999258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 261/528] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:27.000552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.018761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b58188-b2a2-42f9-b475-a3b8a573c202 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.074365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.075523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.075818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.076084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.092804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.095266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:27.128086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:27.130229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 268/529] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:27.132660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.137257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23771576-2f79-4d92-9ed1-4bbb4df3a3a1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.151802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:27.198826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.199729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.200225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.200777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.202501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.203053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.205775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.222139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 262/530] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:26 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.222139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.250904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:27.263719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a2966-ef8a-4ffa-86af-1dbe01c44ac6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.345012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.346201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.346705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.347621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.350261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.351197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.355962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.397146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068200f-a69c-46de-8a55-2d05a441cf31 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.402181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 269/531] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.403076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.469432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.470614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.471113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.471613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.474963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.475531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.478463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.510615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 263/532] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.511520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.549725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:27.616505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7ad54-7652-447b-a873-9f3633b173db None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.624824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:27.654100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8701f6-0cae-42b5-89f7-00ee0288ea6a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.688359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.689602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.690085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.690588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.692559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.693478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.693859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.694711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.695136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.695549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.696307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.696703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=4b9a1a77fd5b40f6b531c79d69044430) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.697175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.698793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.705647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 264/533] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/role_assignments?user.id=4b9a1a77fd5b40f6b531c79d69044430&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.706723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.755329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 270/534] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.769505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:27.844845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29100-c019-44b8-b824-7ff469d994e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:27.887603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:27.888300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.888615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.889016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.889877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=62ae50de406e46dbb2448a8f04426bfe) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:27.890192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:27.891702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe573e-e198-48b4-b554-e118ac8d32a2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:27.905866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 265/535] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/role_assignments?user.id=62ae50de406e46dbb2448a8f04426bfe&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:27.964199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129eb3-6e77-498a-8125-444e36cae0f8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.964888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129eb3-6e77-498a-8125-444e36cae0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.964888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129eb3-6e77-498a-8125-444e36cae0f8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.966372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 266/536] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:27.973540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331847d-6955-4038-94f7-42628b28d705 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:27.973857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331847d-6955-4038-94f7-42628b28d705 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:27.974109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331847d-6955-4038-94f7-42628b28d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:27.984950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:28.011955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b331847d-6955-4038-94f7-42628b28d705 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:28.013702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b331847d-6955-4038-94f7-42628b28d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:28.025485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c81fbc-bbe9-4513-8af3-f5eefca59e91 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:28.088074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:28.095098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:28.098682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:28.098682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:28.098682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:28.182436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 267/537] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:28.230502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331847d-6955-4038-94f7-42628b28d705 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.240590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:28.242733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:28.356830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331847d-6955-4038-94f7-42628b28d705 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:28.391978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:28.410165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:28.411363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:28.411839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:28.420401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:28.441257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 271/538] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:28.448161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.457057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:28.464895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:28.465827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:28.466608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:28.474249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 268/539] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:28.475474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.503740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:28.599821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef04b-c485-4ba1-8d77-01046af8edc0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:28.682803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:28.685929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:28.686446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:28.687198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:28.717072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:28.720684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:28.727113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17465a-b4f4-4829-b99c-de0cca06bf43 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:28.757429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:28.758334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:28.759198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:28.759198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:28.759198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:28.759198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:28.774669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 272/540] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:28.776037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.809407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:28.811462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:28.814979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:28.815515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:28.816564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:28.817080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:28.819085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:28.835318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 269/541] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:28.843891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.903875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:28.924832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f39ed-34d4-43b4-ac0b-ae7f07edc0a8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:28.963811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:28.964441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:28.964728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:28.965004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:28.965753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:28.966062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:28.967549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:28.982128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 273/542] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:28.987916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:28.990265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:29.106157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c6f50-8f48-486b-acec-34267f7f3da1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.120713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf66b3-0a78-4451-b933-8419d7591878 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.144522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.144522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.144522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.144522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.161531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.163138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.173513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.174484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.174837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.175215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.188672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.190347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:29.194040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:29.194907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:29.195751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:29.201999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 274/543] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:29.204376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:29.211877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:29.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:29.242248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:29.244360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:29.244360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:29.244360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:29.244360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:29.248750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 270/544] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:29.254427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:29.362053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7cc5-2e93-4e7a-8ff1-ee5f67b9d40f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.402780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.403470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.403754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.404032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.404953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:29.405267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.406743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.424427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 275/545] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:29.426301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:29.519683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:29.550633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4855f0-a044-415a-9733-7c57ed01b479 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.562903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:29.593438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265efab-8ada-4eb6-a72b-3047631246db None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.649776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.650476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.650759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.651037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.651779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:29.652082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.653527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.657789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.658947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.659455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.659952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.661619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=16969e4969c4457eaec3ed37c5757aa1) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:29.662214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.664637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.674063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 276/546] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:29 2022] GET /identity/v3/role_assignments?user.id=16969e4969c4457eaec3ed37c5757aa1&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:29.678440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 271/547] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:29.683968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:29.836679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9515a0-fc20-4924-bddc-5c01aaa430e0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:29.876235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:29.877110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:29.877972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:29.879121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:29.879709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:29.890618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:29.893081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:29.909289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:29.910434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:29.911327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:29.914140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 277/548] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:29.920237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:30.075513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca9757-3c6a-455f-bf51-b77c5cf2e0fb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:30.128070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:30.128738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.129019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.129290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.130148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:30.130451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:30.131927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0a200-4b4d-44cc-9631-94fe19cb75a6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:30.160500 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 272/549] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:13:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:30.166438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:30.293277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85bb6-0277-4187-883a-3bdb533ec97e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:30.341585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:30.343369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.344147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.344924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.362623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c3d53-a7c4-456e-9f98-e1d759ce543e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.362957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c3d53-a7c4-456e-9f98-e1d759ce543e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.363176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c3d53-a7c4-456e-9f98-e1d759ce543e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.364957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 273/550] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:30.372535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b249-92d0-4d5f-a3b1-a0e9f333b7c2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.372860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b249-92d0-4d5f-a3b1-a0e9f333b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.373527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b249-92d0-4d5f-a3b1-a0e9f333b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.413750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4993b249-92d0-4d5f-a3b1-a0e9f333b7c2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:30.414921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4993b249-92d0-4d5f-a3b1-a0e9f333b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:30.416665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:30.433308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:30.435246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70eb0f-c79b-487b-aa75-2d022f6f69a3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:30.465751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:30.559737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 278/551] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:30.564463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf8d-4911-4384-ae61-22eea00efaa6 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.564463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf8d-4911-4384-ae61-22eea00efaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.564463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf8d-4911-4384-ae61-22eea00efaa6 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.566921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 279/552] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:30.570518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ac94-9828-497d-bbbd-d7007a4887c1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.570518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ac94-9828-497d-bbbd-d7007a4887c1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.570755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ac94-9828-497d-bbbd-d7007a4887c1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.572464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 280/553] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:30.573286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ac94-9828-497d-bbbd-d7007a4887c1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:30.578762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 274/554] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:30.580447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe9b1-1cb3-4aa5-bd74-f1a3aa623bcd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.580950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe9b1-1cb3-4aa5-bd74-f1a3aa623bcd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.581439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe9b1-1cb3-4aa5-bd74-f1a3aa623bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.618045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2ebbe9b1-1cb3-4aa5-bd74-f1a3aa623bcd None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:30.620576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2ebbe9b1-1cb3-4aa5-bd74-f1a3aa623bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:30.698039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:30.722719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ac94-9828-497d-bbbd-d7007a4887c1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:30.740705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 275/555] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:30.744378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.745417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.746773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.772113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:30.772986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:30.773336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:30.773585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:30.777717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:30.779805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:30.786246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:30.787784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:30.815202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:30.816141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:30.817134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:30.817134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:30.817134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:30.817134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:30.820653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 281/556] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:30.823094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:30.904446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 276/557] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:30.904446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:30.929886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:30.985500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42171009-00dc-491c-a1b1-fe8698c122c3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.040849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.041767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.042115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.042454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.053829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.059971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efdcb9-001f-47a4-a327-411f35047053 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.062236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:31.075403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:31.076547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.077497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:31.078306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 282/558] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:31.079571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.110178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.111644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.112121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.112573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.125042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:31.133517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:31.135953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.137629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:31.140723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 277/559] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:31.141748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.233416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.260699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad88056-aa00-41be-aad3-9892585c548c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.267993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5357aa-c4be-4492-8737-2def45239150 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.310365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.311297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.311582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.311867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.312637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:31.312949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.313719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.314373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.314810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.315271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.315729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.329412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 283/560] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:31.330301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.333862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:31.341781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:31.342825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 278/561] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:31.349742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.427506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.472699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaed2b5-d62d-4432-9e28-3607a27d87f2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.481834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcad42-4d29-4867-807e-bf7c211084e1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.520496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.521082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.521361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.521626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.521803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.522335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.522491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:31.522623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.522770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.522898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.523665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:31.523969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.524114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.525403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.527171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.534889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 284/562] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:31.535911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.546765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 279/563] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:31.548676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.661066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c2fe5-8f50-41a6-b7a5-e3d8cb130557 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.687178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.705858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64006f0-f64f-437c-b744-d7c965dfb8f3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.711073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.712470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.712960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.713444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.726155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.728937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.743735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.744438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.744720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.744987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.745777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:31.746081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:31.746619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:31.749179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:31.750974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:31.751821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.751956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 285/564] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:31.751956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.768691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 280/565] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:31.768691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.837307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:31.890065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f311f7-3abc-4f02-85bd-ab868aec9fcc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.899537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203b46b-0e17-4210-b4a0-83aef9ece485 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:31.925804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.926957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.927411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.927854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.940446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.942558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.959650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:31.961012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:31.961483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:31.961945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:31.962169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:31.963014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:31.963871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:31.963871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:31.963871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:31.963871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:31.965742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 281/566] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:31.967296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:31.978391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:31.980021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:31.998422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:32.005762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 286/567] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:32.006882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.036026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.079351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb229d3c-daf2-4896-81cc-937142e9d884 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.090918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.116057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0903d8-d27d-46a6-99ff-7f93f9fa197b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.116577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.117255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.117565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.117824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.118619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:32.118935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.120444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.135231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 282/568] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:32.136093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.157325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.158342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.158763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.159162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.172336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.173989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e15fb748-efed-43f5-883e-5cdf7be88d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:32.193783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:32.194784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:32.195691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:32.195691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:32.195691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:32.195691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:32.195691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 287/569] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:32.196287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8efbc1-55d6-44e5-82da-6f2b20942783 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.196601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8efbc1-55d6-44e5-82da-6f2b20942783 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.196876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8efbc1-55d6-44e5-82da-6f2b20942783 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.198135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 288/570] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:32.198813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8efbc1-55d6-44e5-82da-6f2b20942783 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.252459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8574-cfe4-483d-968f-48baf5fe61c4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.286708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.287631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.288307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.288616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.288886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.289603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:32.289906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.291295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.303469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 283/571] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:32.304400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.306113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8efbc1-55d6-44e5-82da-6f2b20942783 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.349101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.356832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.357887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.358289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.358659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.359484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:32.359886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.361468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.374011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 289/572] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:32.374865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.426721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2711e54-4b8b-4453-a221-e76365ff5101 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.464537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.465362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.465752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.466140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.467338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=47acb0dcc4164628a2442c7dd26d478b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:32.467972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.469757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893cb94-6f81-4f40-b905-5ebd2924450d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.477020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 284/573] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/role_assignments?user.id=47acb0dcc4164628a2442c7dd26d478b&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:32.479005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.479482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.479961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.493057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb237f-0be0-43a4-b8c8-d5841b4be9b7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.506115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:32.508056 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:32.541923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.542609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.542894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.543169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.543923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:32.544232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.546283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.560227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 290/574] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:32.561243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.569955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.617363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 285/575] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:32.618716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.667391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.741481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df28c2-6b0a-45fd-8682-be8bbe0e4a04 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.776473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268ee7-4868-40e5-b4a1-1367d01244ed None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:32.799028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.799841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.800146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.800793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.809743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.811736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.826572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:32.827393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:32.827726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:32.828187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:32.828658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:32.830390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 291/576] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:32.832172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.843371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:32.844937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:32.863368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:32.864461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:32.865831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:32.869664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 286/577] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:32.870568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:32.929820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.990439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:32.999247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eea2a5-42c0-435f-b435-fa5b5adad1a4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.020394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3efa8f-f33a-4c1d-91e0-cd21932658d3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.041288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.042708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.043142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.043561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.053995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.056820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.072258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.073109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.073394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.073674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:33.085881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:33.087745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:33.089635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:33.090473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:33.090606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 292/578] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:33.090606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:33.098438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:33.100789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 287/579] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:33.102900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.241369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1059fe5-ec88-48ea-9c70-a44a1061d9e1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.270391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba11c49-2608-486a-8897-9c459d072e19 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.280646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.281340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.281632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.281911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.282769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:33.283090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.284590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.305760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 288/580] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:33.306627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.314728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:33.317104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:33.333451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.334066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.334352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.334631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.335495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:33.335815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.337336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.365628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 293/581] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:33.366921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.441089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c266a03-1733-49cc-aa19-84fe310a6796 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.495516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.496554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.497000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.497441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.499414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:33.499950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.502307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.526971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bc442-989a-4e6b-949e-a26eb4b73fff None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.537488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 289/582] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:33.537488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.594623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.595820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.596297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.596841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.598344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:33.598813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.601420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.624805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 294/583] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:33.624805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.640523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:33.692445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:33.707091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6581be-5c68-487c-8f03-ac480d0f172d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.755579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.757261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.757440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.757915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.771854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.773899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.782638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a4976-1f20-435f-82cb-9db05255905e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:33.793820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:33.795110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 290/584] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:33.797769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.844385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:33.845185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:33.845468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:33.845748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:33.858737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:33.860296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:33.880832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:33.881885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:33.882749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:33.882749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:33.882749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:33.882749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:33.882749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 295/585] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:33.883486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:33.969448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07169e-77fe-4988-bd28-203ede829c4a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:33.986165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:33.999586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.007007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd62ba-549a-4b59-8439-78c2d90463b1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.025422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.026240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.026524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.026795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.040037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.041813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.047743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.048599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.048882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.049158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.062018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:34.062335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.063304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.064281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:34.064281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:34.064281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:34.064281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:34.064663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.068768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 291/586] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.069702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:34.086453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.087985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 296/587] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.089833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.207856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6c275-74f5-459d-b7aa-c691ce26deee None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.208141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb4f6-0cbd-4f17-99bd-1d07f9af1b8d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.247261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.248023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.248464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.248783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.249564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:34.249879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.251338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.259628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.260942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.261379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.261655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.262412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:34.262718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.264310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.274375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 297/588] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:34.274375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.286583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 292/589] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:34.287230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.290246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.323213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.412661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff9d52-f3cd-4c09-b8aa-480bb76c112a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.418548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07c580-6718-47f8-bd24-b568150dc1ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.466671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.467360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.467360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.467360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.468890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:34.469134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.470495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.471504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.472455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.472746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.473023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.474527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:34.474918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.476262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.488707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 293/590] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:34.489846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 298/591] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:34.490224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.490685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.578695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.597242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.617549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a435-c129-46d4-a231-5e060ff43ac0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.621126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9701b-e9d1-4196-9fb6-a4f5c1a7c0e4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.663343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.664165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.664478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.664756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.665921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.666762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.667029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.667306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.672274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.673814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.674769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.676227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:34.688611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.690420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 294/592] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.692125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:34.693654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.695111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 299/593] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.697004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.801964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111f2cc-c1df-4b8d-991e-06816878a971 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.805633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d16491-54b0-4f6b-bfe5-4a96a5a7ab7a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:34.837245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.845395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.846205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.846533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.846885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.848051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:34.848904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.849190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.849462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.855925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.857105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:34.857710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.858487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ac4d5b61-7b15-4de7-8a5d-c908acb43697 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:34.873445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.874538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:34.875458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 295/594] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:34.876043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:34.876866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:34.877748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:34.878953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 300/595] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:34.880148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc048-3e14-4b0a-bc83-1c4da7c19e20 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:34.880454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc048-3e14-4b0a-bc83-1c4da7c19e20 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:34.880731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cc048-3e14-4b0a-bc83-1c4da7c19e20 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:34.881282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.883131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 296/596] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:34.884154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cc048-3e14-4b0a-bc83-1c4da7c19e20 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:34.888220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:34.997788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b3015-16f1-4ee0-96aa-3141f231d0d8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.012468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cc048-3e14-4b0a-bc83-1c4da7c19e20 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.036633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.037311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.037590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.037862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.038665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:35.038981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.040508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.050816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.051513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.051787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.052060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.052925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:35.053231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.054255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 301/597] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:35.054762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.055133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.073504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 297/598] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:35.074488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.118729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.148494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.180260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799343c-14fa-4919-baab-57a1193a551c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.197276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6c5-5a03-435e-b8c6-fa28d0116217 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.221620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.221921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.222221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.222464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.223260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:35.223571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.225071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b158037-1d77-472d-b640-f626118b5898 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.235182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.235844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.236123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.236436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.237309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:35.237621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.238946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 302/599] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:35.239222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.240248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.240574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.240853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.258331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 298/600] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jun 27 15:13:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:35.259160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.260707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:35.261918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:35.343066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 303/601] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:35.344521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.369519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d4d9-0210-4ab6-a750-4971fa838671 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.396785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.397012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.410072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.410971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.411252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.411522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/4b9a1a77fd5b40f6b531c79d69044430/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.437911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=4b9a1a77fd5b40f6b531c79d69044430, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.439172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.454785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 299/602] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:35 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/4b9a1a77fd5b40f6b531c79d69044430/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:35.455597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.463610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b70d1-f4fc-4b84-847d-ecb81eb68fff None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.510916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.512909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.513229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.514596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.534057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=c3f4d2d7923a4a9a8f59968f770949d6, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.535583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.552394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 304/603] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:35 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:35.557741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.596731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa904776-964a-4dc8-848c-17b72da6aba8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.659620 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.661404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.662044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.662463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/62ae50de406e46dbb2448a8f04426bfe/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.687652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=62ae50de406e46dbb2448a8f04426bfe, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.690429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.708388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.724608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 300/604] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:35 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/62ae50de406e46dbb2448a8f04426bfe/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:35.726160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.733610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af316b-9f66-4997-beab-c48319e5992c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:35.774941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:35.775807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:35.776091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:35.776405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:35.787035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:35.789024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:35.790587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:35.810494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:35.811510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:35.812431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:35.812431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:35.812431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:35.812431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:35.812431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 305/605] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:35.821324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:35.962725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaa571-a2c5-4438-9f1d-5bcbb6ad943d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.001598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d1635-aa58-4717-8321-a4e6bc7172f0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.011863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.012730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.013014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.013366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.030545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:36.039209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:36.040169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:36.041025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 301/606] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:36.049418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.060234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.061099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.061395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.061675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.062550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:36.062862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.064283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.077821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 306/607] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:36.080749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:36.084346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.256726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:36.286579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7804a9-2c2d-4d5d-9dc3-1145a9cc4f76 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.308839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5403f-1948-4296-b733-c058dbcff87d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.342200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.343021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.343311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.343594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.344391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:36.344709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.346144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.356135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 302/608] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:36.364360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.374702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.375719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.376066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.376438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.388475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.390028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:36.411651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:36.412713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:36.413594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:36.418344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 307/609] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:36.425449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.517001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:36.538759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eaaa-ec08-4d9d-b0bd-ffa0046992bb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.569229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869509f-37a4-4466-b1ab-d459f68298d0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.586441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.587503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.587968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.588410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.608655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.610655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.619255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:36.650211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:36.651348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:36.652434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:36.652434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:36.652434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:36.652434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:36.652434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 303/610] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:36.653803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.654523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.654805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.655080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.655887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:36.656196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.657791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.660599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.680035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 308/611] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:36.685601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.807284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40c2b7-655b-45e0-9d79-cdbe7ff36f30 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:36.850540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:36.851501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:36.851936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:36.852373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:36.853414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:36.853860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:36.856408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:36.879551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 304/612] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:36.888045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:36.955546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:37.010506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d8b1e-3e55-4751-81d0-83789f2228d9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.014683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff3545-9a2d-4719-b2c1-b0c48ec517d2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.056261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.058115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.058854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.059357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.069312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.071108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:37.091956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:37.093429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:37.094853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:37.098257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 305/613] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:37.104189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:37.120437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:37.147207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.148234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.150534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.150828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/16969e4969c4457eaec3ed37c5757aa1/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.171535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=16969e4969c4457eaec3ed37c5757aa1, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.173074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.197334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 309/614] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:36 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/16969e4969c4457eaec3ed37c5757aa1/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:13:37.273495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f122f-937d-4563-b4c6-0ddd8ccaeacc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.291785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:37.320992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.321589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.321876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.322152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.323010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:37.323324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.324842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.354718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 306/615] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:13:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:37.357602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:37.501602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fcfb9-6f43-4922-b977-519f109a85c0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.541279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.542322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.542462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.542831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.555150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.556697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:37.577095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:37.578548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:37.580285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:37.580285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:37.580285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:37.580285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:37.580285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 310/616] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:37.586079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:37.698159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9cb28-5b78-4788-964b-651637d14da5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.735696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.736423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.736711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.736984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.737717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:37.738023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.739438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.756401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 307/617] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:37.758013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:37.839720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:37.912290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1cba8-de9e-4423-81d7-b286554c9317 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:37.954565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:37.955399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:37.955898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:37.956038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:37.968156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:37.969774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:37.991280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:37.993034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:37.994773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:38.000349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:38.003083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.049343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:38.131189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9ffb3-6f51-4a68-9f51-fa999d26e3f7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:38.175146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:38.175838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.176132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.180600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.181609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:38.181994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:38.183521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5628e-dcd3-471f-b299-c2e315aaa281 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:38.203987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 308/619] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:38.210120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.353863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd740f-7145-4b87-80b6-cd50ad182c4f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.354204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd740f-7145-4b87-80b6-cd50ad182c4f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.354559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd740f-7145-4b87-80b6-cd50ad182c4f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.356522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 309/620] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:38.362540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb7ef-6c16-4bd9-8679-9295c0666552 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.362818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb7ef-6c16-4bd9-8679-9295c0666552 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.363093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb7ef-6c16-4bd9-8679-9295c0666552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.399205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fa7eb7ef-6c16-4bd9-8679-9295c0666552 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:38.400789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa7eb7ef-6c16-4bd9-8679-9295c0666552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:38.415417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:38.449954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcf558-738b-44e9-8502-e6d327824546 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:38.500652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 310/621] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:38.500652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343471-c084-47f8-93ab-3ffe5e00b15d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.500652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343471-c084-47f8-93ab-3ffe5e00b15d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.500652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b343471-c084-47f8-93ab-3ffe5e00b15d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.500652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 311/622] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:38.502275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:38.503592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.504089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.508363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.508363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.508363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.509422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/47acb0dcc4164628a2442c7dd26d478b/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.533717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:38.535011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:38.542044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=47acb0dcc4164628a2442c7dd26d478b, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:38.543616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:38.563386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 312/623] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:38 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/47acb0dcc4164628a2442c7dd26d478b/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:38.564614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:38.565627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.697692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd5d1f-d7c2-42a0-b297-4a2d57c1edd0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:38.735096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:38.739262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 312/624] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:38.746220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:38.747061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.747397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.747676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.765876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:38.766807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:38.774354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:38.775340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:38.776179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:38.781104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 313/625] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:38.782363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.910380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476155d-9181-4a86-ae41-d6f23f2584d8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:38.920353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288b3a-8c3b-4cee-b902-39fec4bfeb9f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:38.953751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:38.954669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.955021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.955365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.967888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:38.968743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:38.969632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.970095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.970551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.971520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:38.972024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:38.973646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c896ba-9abb-401c-8efc-453d1e8465f0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-267065e0-75e6-4dee-80c1-c72a5a335c7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:38.976722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:38.977684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:38.978518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:38.983239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 313/626] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:38.984590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f30e7a-29b8-4ed7-bb05-1fb76001e7dc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.984873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f30e7a-29b8-4ed7-bb05-1fb76001e7dc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.985218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f30e7a-29b8-4ed7-bb05-1fb76001e7dc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:38.987504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 314/628] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:38.990702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 314/628] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:38.992575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:38.992992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f30e7a-29b8-4ed7-bb05-1fb76001e7dc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:38.993296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:38.993771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.012091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.019012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:39.020639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:39.099110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.123702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 315/629] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:39.125718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.144524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f30e7a-29b8-4ed7-bb05-1fb76001e7dc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.188466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.189292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.189576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.189891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.203571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.205557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:39.225621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:39.226655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:39.227538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:39.227538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:39.227538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:39.227538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:39.232008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 315/630] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:39.233481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.256209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e270a8-9f65-4fc5-89c2-70ff62263b0d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.297138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.297938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.298210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.298570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.299262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:39.299573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.301100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.315214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 316/631] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:39.316359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.320494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.365347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f220-6b05-448d-a698-5da7839ec104 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.372239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.413209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.414724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.415690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.416175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.431968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:39.437039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322106f4-185f-4c59-8bac-0ab2e20b6db8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:39.442854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:39.444181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:39.445036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:39.448015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 316/632] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:39.449646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.479351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.480089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.480514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.481340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.481886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:39.482327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.484003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.504192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 317/633] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:39.505942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.575954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91844d-af97-4a94-b004-ce29a754672e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.607245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.619764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13576-292f-498a-8843-63aedc156475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.621007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13576-292f-498a-8843-63aedc156475 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.621442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13576-292f-498a-8843-63aedc156475 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.621866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a13576-292f-498a-8843-63aedc156475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.630672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c99b7-9f0f-47a5-b78c-7e631e47173c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.634592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a13576-292f-498a-8843-63aedc156475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.636537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a13576-292f-498a-8843-63aedc156475 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.640127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-61a13576-292f-498a-8843-63aedc156475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:39.658002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:39.659003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:39.659860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:39.659860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:39.659860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:39.659860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:39.664940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 317/634] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:39.666103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13576-292f-498a-8843-63aedc156475 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.682393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.683644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.684139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.684680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.685781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:39.687829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.689752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.704672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 318/635] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:39.706076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.783095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a13576-292f-498a-8843-63aedc156475 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.819566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.820415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.820642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429dd18-0d9d-41ca-aa84-3f9d80dd886e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:39.820772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.821025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.829746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.831237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:39.852771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:39.853789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:39.854691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:39.862151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 318/636] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:39.863774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.865708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:39.866414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:39.866736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:39.867012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:39.867762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:39.868069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:39.869643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:39.888180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 319/637] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:39.888180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:39.922463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:39.929235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.000418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8025-45d5-44f8-b392-2bf7c8562c45 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.004922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f5d2e-bfda-4075-82e7-59cbf44c6883 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.037098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.037786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.038062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.038434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.039282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:40.039590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.041174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.047893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.049188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.049513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.049806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.061419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 320/638] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:40.063183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.063549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.064717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2080d722-e3ab-4615-8995-3c08eca65fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:40.083358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:40.084492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:40.085552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:40.085552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:40.085552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:40.085552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:40.088477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 319/639] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:40.090802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d766-779d-4ba1-a829-10f5941ec5f8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.091096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d766-779d-4ba1-a829-10f5941ec5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.091380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d766-779d-4ba1-a829-10f5941ec5f8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.092896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 320/640] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:40.093660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d766-779d-4ba1-a829-10f5941ec5f8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.176362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.195167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef550c7-c524-4940-a9cb-1d338a1574b0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.212040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.233334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d766-779d-4ba1-a829-10f5941ec5f8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.236303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.237152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.237445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.237721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.246060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.247686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:40.264754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:40.265830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 321/641] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:40.267583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.276122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.276833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.277111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.277391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.356785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.358817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.410898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978c7107-be97-4e5c-a515-db5e2e32944c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.464135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.465051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.465323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.465563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.466444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:40.466806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.468845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c599e7-05ea-4e0c-bc47-5a2c24343da2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.482985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.483354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 322/642] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:40.484850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.485138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.485419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.497435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.503701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:40.504209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 321/643] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:40.505120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:40.506129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.621333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea025319-e62d-48aa-a7a5-2e9bea3a4006 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.627794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 323/644] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:40.629188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.692715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.694093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.694520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.694931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.695945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:40.696456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.698068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.718588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 322/645] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:40.723800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.767055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1724f-cb1b-416d-a4a5-23eff4594783 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.783736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.805849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.806647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.806959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.807195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.810608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:40.816500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.818708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:40.835597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:40.836650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 324/646] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:40.837709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.852032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d2c72-405d-4bb8-9c4c-dac98e2d1eff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:40.903159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:40.903909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:40.904684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:40.904684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:40.917993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:40.919531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:40.945609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:40.946611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:40.947481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:40.947481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:40.947481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:40.947481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:40.947709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 323/647] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:40.948615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:40.975015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfdf65-f4f8-4c4c-82e1-41829dd2dc51 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.017337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.018168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.018448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.018728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.031120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:41.039182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:41.040278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:41.041171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 325/648] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:41.041853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.073007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.076574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b7355-4005-4b94-8122-e0cb4ef04029 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.118178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.119581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.120608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.121045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.121491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.179169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a3c9e-0fd6-4390-b1a8-05022d5ed071 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.201106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.203063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd7271-4b5f-49be-b469-20b14c437ea6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.224648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.225335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.225624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.226131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.226782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:41.227091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.228612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.248019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 326/649] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:41.249013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.338632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.347806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 324/650] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:41.350213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e04a6-9615-499e-ba6c-0a403de3f2a2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.350784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e04a6-9615-499e-ba6c-0a403de3f2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.351135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e04a6-9615-499e-ba6c-0a403de3f2a2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.352739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 325/651] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:41.353494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e04a6-9615-499e-ba6c-0a403de3f2a2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.368197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642036a3-1e51-48e8-b219-c3e1178a9899 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.383849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.408461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.409137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.409441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.409687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.410495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:41.410726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.412114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.428082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 327/652] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:41.428815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.475224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e04a6-9615-499e-ba6c-0a403de3f2a2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.519019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.519851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.520137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.520474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.521696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:41.522189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.523918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.535320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 326/653] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:41.536154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.550984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed027-b051-454e-85e8-7165c33f5645 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.601253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.602034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.602402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.602782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.613156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.666391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.669129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733e57f-432d-4c57-a6bb-d1692e4cac30 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:41.696916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.698408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.720166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:41.721330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.721330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.721546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.734042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:41.735473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f656bc13-46fc-4205-b86b-6f5f5dc5601d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:41.753554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:41.754656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:41.755524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:41.755524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:41.755524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:41.755524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:41.755524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 327/654] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:41.756128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:41.756768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:41.756768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:41.775614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:41.776572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:41.829454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 328/655] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:41.830015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.882078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 328/656] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:41.882967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:41.938425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.949491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:41.981284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedea96-ea95-45a0-813c-9050d9e20065 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.002027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a618b-9962-4ea6-b43c-e780c8708734 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.023917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.024821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.025075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.025354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.033257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.034649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.043603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693cba-0723-4986-b458-35177d94e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.044437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693cba-0723-4986-b458-35177d94e452 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.044736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693cba-0723-4986-b458-35177d94e452 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.045078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693cba-0723-4986-b458-35177d94e452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:42.049405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.050281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:42.051168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 329/657] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.052073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.059060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75693cba-0723-4986-b458-35177d94e452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.061003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75693cba-0723-4986-b458-35177d94e452 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-75693cba-0723-4986-b458-35177d94e452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:42.081043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.082272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.083157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:42.083157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:42.083157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:42.083157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:42.087081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 329/658] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.088001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693cba-0723-4986-b458-35177d94e452 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.166353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9755d9a-c0f3-417c-96d3-9e7bb9951cf3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.187034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.194629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693cba-0723-4986-b458-35177d94e452 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.203610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.204337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.204627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.204903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.205653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.205959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.207416 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.209233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.221527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 330/659] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.222407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.238710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.239533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.239820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.240212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.251748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.253362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:42.272667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.273753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.274753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.338650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32491fbc-14e6-4522-af93-a204f46a83f2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.377407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.378212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.378493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.378766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.389317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35268-e2bd-4361-851a-5a42c6d316bc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.390165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:42.397836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.399359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 331/661] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.400185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.427241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.427914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.428196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.428539 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.429330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.429640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.431089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.443475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 331/662] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.444303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.458955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.459350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.504277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c2b3-47c2-44ca-9e59-f774aa058bed None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.538615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.539401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.539685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.539961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.540727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.541053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.542443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.550794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d00ddc-3f9d-479f-a3cd-e81fe5d1ee75 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.555383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 332/663] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.556039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.587609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.588440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.588720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.589003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.595809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.597300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:42.611794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.612703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:42.613601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 332/664] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.614068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.660001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a8162-810f-479b-bbda-228f5843aaa9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.674458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.683925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.694918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.695540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.695830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.696099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.696853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.697160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.698538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.710221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 333/665] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.711027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.723579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115186-dea0-4503-abc9-f2ed774be44a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.757981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.758569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.758847 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.759118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.759876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.760179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.761592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.769801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 333/666] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.770623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.811972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54929ee4-a2b4-4124-a124-90bff3b6fb34 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.847623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.848300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.848589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.848861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.849690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:42.850000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.851452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.858323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 334/667] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:42.859240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.887435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151b79b-ccf7-40d6-ad4f-2a9eaa5a3236 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:42.907803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.929948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:42.938715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:42.939664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:42.940007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:42.940348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:42.952080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:42.953684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:42.971789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:42.972920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:42.973845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 334/668] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:42.974435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:42.991764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59651422-e720-497e-826f-5f4a5a8ea792 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.036803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.037409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.037965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.037965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.038762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.039072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.040600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.053721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 335/669] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.054566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.085498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd04e3a-b4ad-406e-82b3-7ea5331bfd4d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.122508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.123401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.123955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.124036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.131166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.132659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:43.147225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:43.148183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:43.149089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 335/670] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:43.149562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.175727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.180432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e5c20-6f70-4ac3-a38f-a4a93077a976 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.209641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.220215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.221180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.221465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.221758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.233331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.234815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:43.253691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:43.254708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:43.255592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:43.255592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:43.255592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:43.255592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:43.255592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 336/671] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:43.256548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.267006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4522-d829-46a4-8b46-e6e7690e8341 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.302593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.303187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.303470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.303745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.304619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.304936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.306455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.325025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 336/672] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.330159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.366886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ea3b2-56ab-4bb4-9591-4892bad93292 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.404436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.405208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.405483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.405769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.406595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=4b9a1a77fd5b40f6b531c79d69044430) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.406909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.408398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.415594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 337/673] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/role_assignments?user.id=4b9a1a77fd5b40f6b531c79d69044430&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.416191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.433700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.466538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb855-ed85-46ce-aaa3-056c88d3b17f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.482013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.515848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.516939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.517440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.517902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.518983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.519524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.521234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.534065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5299b9-55ae-4bc9-90bd-895e82d5299f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.535497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 337/674] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.536691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.580074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.580835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.581121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.581399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.582194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.582511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.583913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.597909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 338/675] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.599201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.666952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c257604-2a9d-4b02-a7a0-022641091ef5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.689585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.713981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.714940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.715234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.715768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.717817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119e1d6-afa5-47c7-b1a4-f3813b4ac731 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:43.754840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:43.755633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:43.755986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:43.756347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:43.757261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=62ae50de406e46dbb2448a8f04426bfe) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:43.757640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.759166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.765908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 339/676] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/role_assignments?user.id=62ae50de406e46dbb2448a8f04426bfe&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:43.766824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:43.776849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:43.797004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:43.798603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:43.919417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 338/677] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:43.925283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.004839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.051856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8e212-0ee7-44db-be46-5f1276e809e9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.066844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050db4c-c030-4897-aa1d-9e7575bf56d4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.096962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.097806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.098093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.098366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.111171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.112694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.113984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.115156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.115829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.116115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.126125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.127626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.129717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:44.131478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:44.132540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:44.133688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:44.133688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:44.133688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:44.133688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:44.134087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 339/678] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:44.140121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:44.147093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:44.148059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:44.149039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:44.157483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.269030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee0488-11cf-4495-9053-00e08ac09a46 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.296008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.306045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4370673-08d1-4759-a85d-e3389c54ae10 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.315268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.315955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.316240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.316626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.317370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:44.317683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.319457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.332702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 340/680] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:44.341963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.362129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.366102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.366686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.367167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.368386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:44.369608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.372581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.410745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 341/681] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:44.417237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.494469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312df7f-8fa9-415d-8669-7b7ac1040e97 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.531591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.532816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.533320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.533787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.541924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.543762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:44.559575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:44.560782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:44.561752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:44.564434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 341/682] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:44.569341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.571433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.657568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.675114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aed1f-25b5-4519-a76d-d399e843e00d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.738048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.738720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.739005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.739280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.766613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb7886-ff83-4001-8ed4-b6e04b8d981f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:44.847337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:44.848043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:44.848352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:44.848635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:44.849450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:44.856589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.859167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.873940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:44.875438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:44.882069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 342/683] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:44.887695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:44.938767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:44.988949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 342/684] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:44.994561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.108879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e71e8f-45c7-4543-b0bd-f232f0de56a3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.141640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.146047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9804cd-35e8-4910-95f0-c8e1466e0baa None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.156809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.157690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.157977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.158285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.182433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.187820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:45.225855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:45.226848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:45.227785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:45.227785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:45.227785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:45.227785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:45.232846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 343/685] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:45.234428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.235113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.235392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.235668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.236538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=16969e4969c4457eaec3ed37c5757aa1) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:45.236855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.238411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d65a2-cf87-4d9d-9db9-7207ab8a40a2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.242611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.252197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 343/686] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:44 2022] GET /identity/v3/role_assignments?user.id=16969e4969c4457eaec3ed37c5757aa1&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:45.259629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7ca5-5728-48b0-bb32-0b1af1e75fa4 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.259975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7ca5-5728-48b0-bb32-0b1af1e75fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.260271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c7ca5-5728-48b0-bb32-0b1af1e75fa4 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.261936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 344/687] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:13:45.262266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.268530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.268877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.269091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.290991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:45.292441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:45.391182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96bb9a-bab9-4fb5-a86d-ffd85ad4ddcb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.429889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.430712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.431276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.431651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.432533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:45.432950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.434496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.448744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 344/688] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:45.456443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.479348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 345/689] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:45.505063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.531259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.570742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82600da4-27c8-46d6-9051-eb60ccae94ea None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.608101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.609046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.609395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.609735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.617995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.618160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.620534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.626154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b123a-2d0c-4c18-a1f7-d1fe0bcea088 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:45.637769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:45.638871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:45.639883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 345/690] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:45.645532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.665714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.666902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.667374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.667827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.679977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:45.688460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:45.689401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:45.690219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:45.693615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 346/691] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:45.699488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.759121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35f607-1b20-4c59-a976-c46772280ded None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.795931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.806923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.807947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.808776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.809281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.810304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:45.810944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.813442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ea80d-7d0e-4977-848d-df0b52f10364 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.814453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240a66-7424-489d-b2c5-fdad276290f3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:45.830050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 346/692] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:45.833750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78886e14-009a-43fe-9e91-870b325b34f4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.834037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78886e14-009a-43fe-9e91-870b325b34f4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.834314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78886e14-009a-43fe-9e91-870b325b34f4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.840427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 347/693] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:45.841831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78886e14-009a-43fe-9e91-870b325b34f4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:45.877997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:45.879780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.880627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.881178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.882881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:45.883682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:45.886070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb017-6c51-47cf-beb4-1800142e69fa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:45.899195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 347/694] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:45.901678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e443-872a-4963-b8ff-13f95263133e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:45.902195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e443-872a-4963-b8ff-13f95263133e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:45.902996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e443-872a-4963-b8ff-13f95263133e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:45.927295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3412e443-872a-4963-b8ff-13f95263133e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:45.930177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3412e443-872a-4963-b8ff-13f95263133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:45.946409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:45.963891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78886e14-009a-43fe-9e91-870b325b34f4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.003282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.003944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.004228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.004551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.005393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=47acb0dcc4164628a2442c7dd26d478b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:46.005807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.007164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.020801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 348/695] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:45 2022] GET /identity/v3/role_assignments?user.id=47acb0dcc4164628a2442c7dd26d478b&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:46.021847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.037684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 348/696] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:46.055974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e443-872a-4963-b8ff-13f95263133e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.165158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.223385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e443-872a-4963-b8ff-13f95263133e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.243115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c813-b3c0-4a12-ad9d-91847c3477ec None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.269250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.270020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.270304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.270574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.271290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.272655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:46.288132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:46.289180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:46.290258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:46.291237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.291353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 349/697] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:46.291437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.291437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.291437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.301797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.310876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.313000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.317323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:46.341266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:46.342442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:46.343720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:46.343720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:46.343720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:46.343720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:46.343720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 349/698] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:46.348871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.440556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5dcf-30ad-49e0-91ed-fccbc7e46d44 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.463120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0eb4e-9cd5-45d9-9d33-c7e3e00dee4f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.465936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.493171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.493737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.494020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.494288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.495280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:46.495590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.497190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0a60a-ee0e-47ee-96d0-5ccff7746db3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.504772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 350/699] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:46.505182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50c804-2532-445c-8727-b5f853773e24 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.505468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50c804-2532-445c-8727-b5f853773e24 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.505746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50c804-2532-445c-8727-b5f853773e24 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.507215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 351/700] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:46.509093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.510067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.510564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.511097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.512088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:46.512642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.512764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.513059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.513338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.515079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.532903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 350/701] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:46.534215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.534324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:46.535427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:46.624131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 352/702] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:46.625107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.629292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.642745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a0a5f-91f7-43b1-90db-cfc22ff9af00 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.680958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.682042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.682540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.683021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.683953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:46.684498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.686537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.700480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 351/703] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:46.701915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.747023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3f690-5785-4d95-8d8b-33de3b84c7e3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.770643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.791194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.791979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.792259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.792567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.802122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.803680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:46.822955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:46.823995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:46.825574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 353/704] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:46.837968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:46.899512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311e699-8168-4eba-b25c-5d93fc655141 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:46.939121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:46.940279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:46.940837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:46.941313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:46.954070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:46.955982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:46.977626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:46.978634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:46.979564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:46.979564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:46.979564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:46.979564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:46.979791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:46.982561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 352/705] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:46.987796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:47.032697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc8236-508a-4b84-9d62-e889df5957a8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:47.072724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:47.073372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.073655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.073931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.075085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:47.075085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:47.077373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:47.092459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:47.102427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 354/706] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:47.107680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:47.117166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7308b15-3acd-472b-82ba-e3dc8b16f426 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:47.187267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:47.188837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.189326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.189816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.190813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:47.191467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:47.195024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:47.221361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 353/707] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:47.226902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:47.303872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ef47b-b999-4265-ad93-fb97d8d3234b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:47.340965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:47.360189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:47.362635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.363921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.364979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.449705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1b2eb-f54a-41de-854b-1c055f20976e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:47.501797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:47.504605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f59622-7f00-48a1-807c-589fd8b154eb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:47.519741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:47.520845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.521350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.521827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.522761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:47.524455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:47.544192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:47.563024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 354/708] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:13:47 2022] POST /identity/v3/users => generated 254 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:47.675885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 355/709] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:13:47.685590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:47.846234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35119f5b-07a6-4205-8c1a-482ca990b031 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:47.854258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823994-2feb-43f5-988d-7b4eb02234a5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.854899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823994-2feb-43f5-988d-7b4eb02234a5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.855453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823994-2feb-43f5-988d-7b4eb02234a5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.864672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 356/710] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:47.866432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.867034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.867531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.886142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:47.887882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:47.898682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:47.899884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:47.900558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:47.900989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:47.914413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:47.915207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:47.918278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:47.940463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:47.941386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:47.942263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:47.942263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:47.942263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:47.942263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:47.945243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 355/711] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:47.951569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:47.991191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 357/712] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:48.018930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.023112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:48.080071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64acda-daf4-4a65-a015-085916d032a3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.115608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.116722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.117248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.117756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.118699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:48.119209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.120836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.134580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 356/713] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:48.138319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece081e7-f70b-4154-931b-d31c66448cef None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.141227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.177244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.178070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.178438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.178724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] PATH_INFO: `/v3/services/network` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.179441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.180996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:48.194773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:48.196160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:48.197129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:48.200462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 358/714] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:48.202712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.224630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:48.283419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38ec4d-f3d8-48b7-9098-f939403be4e8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.303604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:48.342058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.343843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.344648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.345388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.357751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.360671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:48.385893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:48.387496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:48.389200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:48.390872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e543b-04aa-4832-b43b-13367e78c379 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.394273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 357/715] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:48.403467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.454798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.455446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.455722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.455997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.456760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] RBAC: Adding query filter params (name=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:48.457071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.458596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.472997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 359/716] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:48.479577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.534078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b7ce6-68db-48e5-a67c-381a376fa61c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.603005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.604067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.604581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.607646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.608948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:48.609469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.611145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.628199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 358/717] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:48.634436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:48.639094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:48.666699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c6d5-8a59-488e-9cbb-59440e184028 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.681555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:48.741121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.741712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.741992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.742262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.742984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:48.743293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.746065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9967e11-c5f8-4f38-a2b8-cf7830227fe4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.758463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 360/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:48.801473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48c3d-82e5-40f0-99a9-0cbb95e5c2ad None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:48.843278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:48.843965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.844250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.844557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.845410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:48.845722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:48.847267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb546bd-5310-41aa-ac89-0bf1845c4700 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:48.854415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 359/719] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:48 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:13:48.881352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146111c2-7b7f-4e52-b795-8e6f941ab073 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.881790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146111c2-7b7f-4e52-b795-8e6f941ab073 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.882169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146111c2-7b7f-4e52-b795-8e6f941ab073 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.884127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 361/720] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:48.890650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:48.891182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:48.891616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:48.914810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:48.916386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:49.010863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 360/721] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:49.038213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146111c2-7b7f-4e52-b795-8e6f941ab073 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:49.180112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146111c2-7b7f-4e52-b795-8e6f941ab073 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:49.216119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:49.217686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.217972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.218245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.233616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7a7bc509-06a3-4c69-998c-c809a60b94f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:49.241175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:49.242113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:49.243373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 362/722] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:49.250387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:49.277628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95077-bb2c-4ba9-8f69-9102e974eece None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.277875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95077-bb2c-4ba9-8f69-9102e974eece None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.278148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95077-bb2c-4ba9-8f69-9102e974eece None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.280297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 363/723] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:49.288510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:49.292487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.293000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.293837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.310034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:49.325344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:49.327898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:49.438474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f2c6-ddfa-43fd-a958-78972e11a25a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:49.483360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 364/724] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:13:49.497307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:49.498034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.498358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.498960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.499466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:49.499779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:49.501230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:49.509410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:49.517319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 361/725] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:49.522610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:49.668786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:49.695493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee56725-1f89-482b-9340-bc6f0646eff2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:49.699461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370bbf1-1e47-464b-a012-97f4c5f41ae8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:49.753030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:49.753846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.754129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.754408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.764507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:49.764827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:49.765110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:49.765374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:49.766118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:49.767543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:49.770968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:49.778357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:49.780259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:49.794938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 365/726] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:13:49 2022] POST /identity/v3/services => generated 251 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:49.813939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:49.814918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:49.815840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:49.815840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:49.815840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:49.815840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:49.815840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 362/727] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:13:49.821777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.002755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0860ce-a4d1-431c-a7f8-a41a4af28075 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:50.039088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:50.039765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.040045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.040335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.041080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:50.041385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:50.042781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:50.056586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 366/728] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:50.062374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.188290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d876b6-95f5-4cea-b8d6-b30493c1591f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:50.249743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:50.250573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.250853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.251126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.254406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:50.265550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:50.267043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:50.284866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:50.285857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:50.286776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 363/729] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:50.291913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.375683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:50.448681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49fca8-3fa0-4ddc-9405-9f6e45af1b11 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:50.503037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:50.503826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.504064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.504365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.505280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:50.505537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:50.507015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06017091-1c8a-4b35-a806-cbaa7398a46d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:50.536466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 367/730] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:13:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:50.541848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.566885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c715d9-e089-4e71-b168-1550ca09b228 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.567231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c715d9-e089-4e71-b168-1550ca09b228 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.567434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c715d9-e089-4e71-b168-1550ca09b228 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.569623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 368/731] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:50.575324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.576695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.576695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.609403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:50.610631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:50.666794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b77ff-5b64-48e6-83a8-70ae763ae3f4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:50.703854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:50.704997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:50.705528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:50.706061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:50.778348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:50.780405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:50.792882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:50.828343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 369/732] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:50.901766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.926495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 364/733] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:50 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:13:50.928173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:50.992711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.032930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ea7a8-e3f3-4791-94c0-fcac0ad63420 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.036786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.071698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.072926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.073415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.074024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.086979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.089109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.103978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3cba7-4ddb-43bb-a624-9d70ee2b00a3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:51.130778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:51.131982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:51.133770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:51.133770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:51.133770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:51.133770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:51.145065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 365/734] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:51.149220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.150596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.151136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.151745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.153771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.182559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:51.200537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:51.201921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:51.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:51.214811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 370/735] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:51.218880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.354691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979db7e0-7680-4b89-9d39-01d1666012e6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.377691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.382120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab17b5e-c232-4d32-bc6a-60bea9eed191 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.398085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.399255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.399742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.400171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.401090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:51.401631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.403314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd718174-ca62-4da9-b105-ad5673294ed2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.421360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.421360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.421686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.421873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 366/736] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:51.422069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.422933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:51.423357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.423961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b263-7b4c-4d23-b5f0-14898a0bd29e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.424438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b263-7b4c-4d23-b5f0-14898a0bd29e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.424871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285b263-7b4c-4d23-b5f0-14898a0bd29e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.425232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658b56d-c23a-4c54-8fb3-89cd5af97aa2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.430945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 367/737] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:51.432231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b263-7b4c-4d23-b5f0-14898a0bd29e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.436358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 371/738] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:51.438269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.438759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.439280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.450073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.461183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:51.462907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:51.552295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 372/739] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:13:51.553992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.585232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285b263-7b4c-4d23-b5f0-14898a0bd29e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.625868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.626767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.627161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.627537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.635440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.637076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:51.652844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:51.653977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:51.655117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:51.661072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 368/740] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:51.661072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.667827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.680593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af992cf-5fe8-423c-a234-670e63437f5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.718219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2415644-2172-4943-a040-1757c33732fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.719027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415644-2172-4943-a040-1757c33732fe None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.719312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415644-2172-4943-a040-1757c33732fe None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.719595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2415644-2172-4943-a040-1757c33732fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.730058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.730903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415644-2172-4943-a040-1757c33732fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.733177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2415644-2172-4943-a040-1757c33732fe None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d2415644-2172-4943-a040-1757c33732fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:51.751497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:51.752462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 373/741] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e205-1c38-4e33-a255-35cb0158cb83 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e205-1c38-4e33-a255-35cb0158cb83 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500e205-1c38-4e33-a255-35cb0158cb83 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.758621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 374/742] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:51.759124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500e205-1c38-4e33-a255-35cb0158cb83 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.773456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a163a8-56d2-4ddc-94c4-8b3b416bb0dd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.810871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.811735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.812023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.812301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.813055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.814441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.830134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 369/743] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:13:51 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:51.830958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:51.913456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500e205-1c38-4e33-a255-35cb0158cb83 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.965136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:51.965952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.966640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.967005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:51.968575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:51.968972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:51.971187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08043-0908-4213-9447-685f0c1cadad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:51.975933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf3f86-5c00-479e-8879-9bbc86e8daf4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:51.981494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:51.992503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 375/744] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:51.994339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:51.994891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:51.995172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.014210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:52.015451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:52.022481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.038025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.039231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.039231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.039231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.040427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:52.040427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.041730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.056763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 370/745] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:52.062255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.112390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 376/746] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:52.112390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.219630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3d381-ccce-431d-ac40-c99114c17f28 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.230792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8533ab-1672-4f8c-9aed-6ec49e907db9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.259765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.268897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.269715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.270095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.270392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.270531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.271465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=47acb0dcc4164628a2442c7dd26d478b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:52.271624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.271971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.272178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.272616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.273799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.281141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.288354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 371/747] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/role_assignments?user.id=47acb0dcc4164628a2442c7dd26d478b&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:52.288354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.288639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:52.322873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:52.323886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:13:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:52.331867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 377/748] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:52.336221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.409100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.484962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc14fff-951e-4fc8-8fb9-03db8210ce84 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.487479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a08c0-94e4-4a63-be1e-ffc7d0322094 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.525443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da905634-f09a-4aba-93da-03a008302425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.526238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905634-f09a-4aba-93da-03a008302425 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.526589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905634-f09a-4aba-93da-03a008302425 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.526936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905634-f09a-4aba-93da-03a008302425 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.528071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da905634-f09a-4aba-93da-03a008302425 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:52.528554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da905634-f09a-4aba-93da-03a008302425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.530408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da905634-f09a-4aba-93da-03a008302425 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.530714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.531718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.532221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.532689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.547678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:52.556179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 378/749] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:52.556726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:52.557597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:52.558443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:52.561024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1269c2-65e3-4526-9852-083d5bbceb5f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.561487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1269c2-65e3-4526-9852-083d5bbceb5f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.561945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1269c2-65e3-4526-9852-083d5bbceb5f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.562954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 372/750] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:52.563961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.564164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 379/751] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:52.565222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1269c2-65e3-4526-9852-083d5bbceb5f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.624812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.679442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1269c2-65e3-4526-9852-083d5bbceb5f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.680001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c85e73-8fac-4b8e-9df4-c09af505b02a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.705237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.717346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.718486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.719015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:52.719138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.719618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.719720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.719934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.720208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.721287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:52.721818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.723438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6776d-5579-4842-832c-9128624e6ed2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.733219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 380/752] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:52.734845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:52.735389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:52.735884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:52.753388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:52.754932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:52.805909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:52.807665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:52.827641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 381/753] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:52.828957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.894622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:52.964514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 373/754] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:13:52.965394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:52.978717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de54eb-aed8-4dea-a0ad-4897e3a44457 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:52.988735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.022521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.023410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.023692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.023967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.038350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.040733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:53.067426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:53.069003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:53.069869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:53.069869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:53.069869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:53.069869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:53.072604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 382/755] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:53.073485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.105453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aeb844-1064-425b-94f6-ba98be806b63 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.149976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.151062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.151353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.151629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.152291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.154592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:53.173636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:53.174629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:53.175450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:53.180428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 374/756] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jun 27 15:13:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:53.181357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.195963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.208389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcddbb0-4bb0-4c47-93f1-fc6181b38ccb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.245068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.246219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.246748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.247224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.258556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.260396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.278327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:53.282181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:53.283553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:53.284480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:53.284480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:53.284480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:53.284480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:53.286597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 383/757] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:53.287674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.305686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428165-bb7d-4475-8ffb-78367d11e741 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.344872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.345562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.345849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.346131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.346874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:53.347184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.348631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.366826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 375/758] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:53.367845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.402006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7e1-9161-47c0-9dd5-de6020d03ccc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.441199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.442315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.442819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.443658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.444720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:53.445245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.447005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.453762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 384/759] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:53.454967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.467110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.488244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8583bf-640e-4c6b-bd2d-b5117aa3b34b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.527506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.528633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.529097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.529571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.530457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:53.530922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.532574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.535423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.547854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 376/760] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:53.548973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.571180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671ec2f-0246-4b78-b284-167623f965e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.611163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.612153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.612562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.613065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.621897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.624420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:53.644415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:53.645573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:53.646867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:53.653764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 385/761] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:53.653764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.689453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073626c-74b5-43f1-b97f-c13a9d9b58e7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.731707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.732939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.733264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.733644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.734471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:53.734887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.736424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.740506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.748621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 377/762] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:53.749715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.790554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c283c-9385-41d2-820f-89013fa9c56c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.829235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:53.842120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.842945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.843445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.843726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.858543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.860555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:53.877271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:53.878243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:53.879124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 386/763] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:53.880086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.888789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec12df1-90c8-451d-ab07-c9321f929f37 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:53.929004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:53.930215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:53.930754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:53.931263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:53.932447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:53.933021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:53.934770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:53.956259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 378/764] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:53.957497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:53.991839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd2f9d-9772-4277-871c-2fd3843330a3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.023659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.034804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.035479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.035762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.036036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.036883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] RBAC: Adding query filter params (interface=public, service_id=7e9367bd18c94687b41b5597b9a02ce8, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:54.037183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.038615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.052173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 387/765] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/endpoints?service_id=7e9367bd18c94687b41b5597b9a02ce8&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:54.052173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.083565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecdd55-d5be-44cd-9083-a7f8ff8d7004 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.107731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.129075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.129759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.130044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.130315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.131145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:54.131455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.132971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aac9c8-ee12-43ef-8245-2c2270df1d12 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.145894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 379/766] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:54.147245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad18f0-7f3a-4911-a925-2e3db9bfed65 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.147531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad18f0-7f3a-4911-a925-2e3db9bfed65 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.147814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad18f0-7f3a-4911-a925-2e3db9bfed65 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.152354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 380/767] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:54.152354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad18f0-7f3a-4911-a925-2e3db9bfed65 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.176739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862739da-f140-4c23-91fc-9f945cea3081 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.218190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.219060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.219356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.219662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.232227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.233991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:54.253214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:54.254312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 388/768] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:54.256866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.275848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad18f0-7f3a-4911-a925-2e3db9bfed65 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.308964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.319166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.319854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.320139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.320452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.321151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.322556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27513a43-43c3-4749-9381-465708e9eff8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.328634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 381/769] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:54.329937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b986-baa4-4738-bc23-49deb69ab65c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.330225 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b986-baa4-4738-bc23-49deb69ab65c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.330505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99b986-baa4-4738-bc23-49deb69ab65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.349867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8c99b986-baa4-4738-bc23-49deb69ab65c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:54.351121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8c99b986-baa4-4738-bc23-49deb69ab65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:54.401042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f654d0-c20d-4ab0-b6ba-71c99360e277 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.409828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.445709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.446731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.447014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.447292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.448153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=5407b0778630459ba6a046d63d69cd02) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:54.448521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.450014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.460685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 389/770] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/role_assignments?user.id=5407b0778630459ba6a046d63d69cd02&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:54.460685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.469638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 382/771] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:54.471563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7879-2164-4e22-8db3-0f824ea9397c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.472092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7879-2164-4e22-8db3-0f824ea9397c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.472551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7879-2164-4e22-8db3-0f824ea9397c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.474645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 383/772] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:54.480449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd4db7-1097-40b1-babf-996647470812 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.480742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd4db7-1097-40b1-babf-996647470812 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.481019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd4db7-1097-40b1-babf-996647470812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.514845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-88cd4db7-1097-40b1-babf-996647470812 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:54.515766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-88cd4db7-1097-40b1-babf-996647470812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:54.613001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7710-7e7d-4a39-9fcf-f3ec24356f31 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.624696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 384/773] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:54.624900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.626189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd4db7-1097-40b1-babf-996647470812 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.655545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.656294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.656614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.656894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.658462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:54.658881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.660936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.678556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 390/774] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:54.679667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.782091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd4db7-1097-40b1-babf-996647470812 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.784872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.804018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728e89a-3609-461c-b6c3-ecb68b180e31 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:54.836735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.838146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.838455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.838737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] PATH_INFO: `/v3/services/network` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.839716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:54.841999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:54.847894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:54.847894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:54.847894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:54.848471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:54.861396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:54.862194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:54.863259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:54.864100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:54.876876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 385/775] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:54.877065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:54.879122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:54.881155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:54.882635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.882781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 391/776] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:54.882781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:54.904362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:54.983761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa73fbe-8ba2-4e32-9916-f5d6be34d6c4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.025220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a59956-a9a6-4d80-9af9-3ff4f503fdcd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.026420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.027101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.027387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.027666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.028426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.028739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.030201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.035769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 392/777] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.037751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.075270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.076335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.076628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.077082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.086424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.088233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:55.107566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:55.110123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 386/778] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:55.112255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.145207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841f58c-e239-43df-9120-66301fdea6c8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.148779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.149540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.186213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.187025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.187320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.187604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.188535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.188946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.190380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.201761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 393/779] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.202717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.239397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062a6c4-80cf-4921-9469-8f115edf7ce4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.282642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.283331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.283614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.283892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.284645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] RBAC: Adding query filter params (type=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.285401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.286807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.296415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 387/780] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.297305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.320334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa0b3-6b0d-43eb-af97-8c257d37cb25 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.362497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.363437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.363721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.363995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.365020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.365689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.367579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.382992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 394/781] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.383915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.402983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.406519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.419902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36875685-8113-40bf-b5d6-b29b199bdeca None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.463787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.464661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.465114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.465455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.482179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.483668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.507763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d201b-9583-46e5-9bfa-acb4241f5aa1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:55.508998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:55.510966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 388/782] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:55.512470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.552999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.553707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.553996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.554280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.555083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] RBAC: Adding query filter params (interface=public, service_id=bb895f539be04257b015e252dd112231, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.555399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.557439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.566219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 395/783] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/endpoints?service_id=bb895f539be04257b015e252dd112231&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.567213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.640885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58885a-1c9a-42ab-9243-1678850a9e65 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.658436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.679920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.680860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.681146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.681430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.683440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=c3f4d2d7923a4a9a8f59968f770949d6, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.684947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.694487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f75f0-fe9d-4c88-afe7-323b126f9219 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.705713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 389/784] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:55 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:55.706773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.710940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:55.744244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.745583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.746057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.746517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.747493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:55.748051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.750032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.765055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 396/785] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.777475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:55.855268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b5c0-04d3-4364-9bb2-f7e2b416f419 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:55.909567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:55.911161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:55.911448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:55.911731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:55.912460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:55.913879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:55.924310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 390/786] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:55.970278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3859ea-0c4f-4a5e-b09f-cf429614e6b2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:56.007568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:56.008483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.008775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.009039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.012306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:56.018423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:56.019899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:56.035338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:56.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:56.037326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 397/787] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:56.044545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:56.156842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:56.206401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc0881-d7f5-4de6-a1aa-4f36af10cce4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:56.258041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:56.258750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.259040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.259323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.260184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:56.260524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:56.261984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:56.289805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 391/788] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:13:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:56.295694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:56.449507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdf52d-8f69-494a-a384-f28b9535fd62 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:56.506252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:56.507071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.507352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.507625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.525540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:56.527027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-aa4c2ec2-4b3a-400f-86d1-cf09721261df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:56.548606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:56.549633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:56.551061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:56.551061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:56.551061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:56.551061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:56.551061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 398/789] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:56.557038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:56.702256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07cc6b-eb5e-4dbc-b713-caa8fe2ef920 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.703252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07cc6b-eb5e-4dbc-b713-caa8fe2ef920 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.703733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07cc6b-eb5e-4dbc-b713-caa8fe2ef920 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.705805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 399/790] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:56.710896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b13592-6b18-437e-bdf1-9e824c5208ae None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.711177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b13592-6b18-437e-bdf1-9e824c5208ae None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.711459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b13592-6b18-437e-bdf1-9e824c5208ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.730950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb2903-700b-4b48-8858-45117c1e1faf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:56.742490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a9b13592-6b18-437e-bdf1-9e824c5208ae None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:56.743845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a9b13592-6b18-437e-bdf1-9e824c5208ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:56.779808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:56.799006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:56.799693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.799977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.800254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.801022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:56.801333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:56.802719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed9345-c476-4276-a0e6-2b3c66d91fbb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:56.821893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 392/791] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:56.821893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abc86-595c-4e6f-81fc-1f5bc45bbb71 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.821893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abc86-595c-4e6f-81fc-1f5bc45bbb71 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.821893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527abc86-595c-4e6f-81fc-1f5bc45bbb71 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.822284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 393/792] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:56.824723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:56.825554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527abc86-595c-4e6f-81fc-1f5bc45bbb71 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:56.867761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 400/793] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:13:56.872377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:56.872377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:56.872377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:56.899591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:56.900819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:56.975836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 401/794] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:56.981078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:56.985658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527abc86-595c-4e6f-81fc-1f5bc45bbb71 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.036726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.037926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.039011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.039519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.049209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.051379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:57.072097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:57.073214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:57.074245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:57.081676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 394/795] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:57.081676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.122886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0641fa-d79e-48d0-b48c-921f34147154 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.136810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.141811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.181013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.182636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.184186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.184779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.185784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.187494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:57.206882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:57.207907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:13:57.208777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:57.214981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 402/796] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jun 27 15:13:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:57.214981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.232090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb3053-f04f-4254-8f93-d2600e0a019d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.269106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.269928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.270213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.270485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.282663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:57.290656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:57.291606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:13:57.292466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:57.296478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 395/797] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:57.297990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.338296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1876c-393a-4a2c-8b93-920007b3de99 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.379513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.380759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.381250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.381728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.382862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:57.383391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.385280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.405689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 403/798] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:57.407270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.409483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4021e-7273-40aa-a3e7-bbb8ac5a0cbe None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.413266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.447484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.450585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.451625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.452014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.452214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.452989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:57.453299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.455458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.465760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 396/799] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:57.465760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.537244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60359c-4e59-4e81-ac3d-3e2369065e1f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.576213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.576788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4488ca9-c983-438f-9a4a-b2c2999f8e90 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.577544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.578023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.578585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.579681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:57.580188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.581817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.596838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 404/800] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:57.598149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.615072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.616112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.616439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.616721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/47acb0dcc4164628a2442c7dd26d478b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.619059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=47acb0dcc4164628a2442c7dd26d478b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.620635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.640736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 397/801] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:57 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/47acb0dcc4164628a2442c7dd26d478b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:13:57.641228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.698419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.714384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:57.723763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940485ab-87af-4683-b5f8-5210937f6607 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.764675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41792-5237-4808-9033-33065375791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.766015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41792-5237-4808-9033-33065375791e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.766508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41792-5237-4808-9033-33065375791e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.766986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41792-5237-4808-9033-33065375791e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.767916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41792-5237-4808-9033-33065375791e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:57.768723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41792-5237-4808-9033-33065375791e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.770462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41792-5237-4808-9033-33065375791e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.776937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 405/802] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:57.783424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41792-5237-4808-9033-33065375791e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.845357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7152aa4-41ea-412a-9557-acfb6f4c7e70 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:57.887002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:57.887836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:57.888117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:57.888494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:57.914443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:57.915957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:57.949291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:57.950266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:57.951277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:57.951277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:57.951277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:57.951277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:13:57.951277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 398/803] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:57.960226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:57.975309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41792-5237-4808-9033-33065375791e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.014174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.015297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.015878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.016374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.017336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.019126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.044968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 406/804] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:13:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:13:58.053451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.077307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21582abe-ef1e-414a-9e59-b912b4b3654f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.131472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.132087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.132401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.132733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.133486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:58.133846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.135318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b933c-4985-4e4e-9be4-8fb3a7132403 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.149708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 399/805] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:58.155729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.174971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb3e4-4188-4cf2-aaee-659684315d36 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.175344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb3e4-4188-4cf2-aaee-659684315d36 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.175597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb3e4-4188-4cf2-aaee-659684315d36 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.180412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 400/806] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:58.183274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00659-6b54-45ac-bbd2-4f79f022482e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.183566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00659-6b54-45ac-bbd2-4f79f022482e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.183848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00659-6b54-45ac-bbd2-4f79f022482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.189484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.203458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ade00659-6b54-45ac-bbd2-4f79f022482e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:58.205777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ade00659-6b54-45ac-bbd2-4f79f022482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:58.291940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 401/807] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:13:58.293458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31adb0-8bd6-4730-991f-5fcdb9fff128 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.293753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31adb0-8bd6-4730-991f-5fcdb9fff128 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.294033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31adb0-8bd6-4730-991f-5fcdb9fff128 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.295732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 402/808] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:13:58.313489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974fb55-645f-43ff-985e-e3ddf2b01ff9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.314424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.314720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.315001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.338199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:13:58.339966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:13:58.359488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.360616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.360908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.361190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.371674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.384432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:58.390878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:58.391853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:58.392821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 407/809] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:58.393407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.493727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.525222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb751f-6cfd-4eb2-bb5e-01de437c1731 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.555511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.566599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 403/810] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:13:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:13:58.568405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.569074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.569974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.570269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.570537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.593566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:13:58.602976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:58.603893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:58.604790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 408/811] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:58.605564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.699052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7b70-bf00-40a5-a850-c5d48bea841f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.734446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.735135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.735419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.735695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.736589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:58.736902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.738368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.750170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a058ea9-4719-4268-b2e5-ebfc80e392a2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.760374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 404/812] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:58.760374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.795666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.796843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.797133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.797412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.798333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.799904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.802451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:58.815492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:58.816717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:13:58.817565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:58.822695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 409/813] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:58.823765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:58.841662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:58.920165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43862b9-c667-49e8-9bd9-6fe7945714a7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.948448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af68-fb5c-4568-9407-9a0add1117d5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:58.969004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.970578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.971359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.971843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.973528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:58.974181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:58.976190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:58.985178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:58.986346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:58.986699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:58.987065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:58.988956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 405/814] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:58.989964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.005425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.007647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:59.030626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:59.031568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:59.032507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:59.032507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:59.032507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:59.032507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:59.036685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 410/815] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:59.037656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.100786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.139198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feded343-c85f-4a99-a582-63d2c5ab6549 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.159531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.175607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcabca-8888-416b-9ea7-d854135d977a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.180978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.181745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.182121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.182471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.183263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:59.183643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.185164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.191161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 406/816] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:59.192155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.215834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.217703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.218174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.218655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.231090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.233091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:13:59.252446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:59.253380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:59.254298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:13:59.254298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:13:59.254298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:13:59.254298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:59.260841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 411/817] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:59.261889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.299807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b001e33-7e06-456e-85a5-258fff0db6b3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.337496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.337848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.338237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.338614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.339478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:59.339915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.341833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.359741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 407/818] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:59.360947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.378363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6d3d-fe5d-4870-ae44-c5ad97d49599 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.380390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.409735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.418673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.420137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.420716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.421201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.422089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] RBAC: Adding query filter params (type=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:59.422677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.424367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.430537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 412/819] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:59.431705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.474593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d817c-3519-4513-ae97-100dd1c3577a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.513503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.514204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.514492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.514814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.515770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:59.515954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.517409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.535199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 408/820] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:59.535199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.548546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319150-7e56-479a-ac22-5495beb6e528 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.585972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.587048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.587515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.588176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.596227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.598006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:13:59.614131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:59.615079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:13:59.615944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:59.618718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 413/821] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:59.619724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.655177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.656645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:13:59.660224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa005c-a5d4-4072-a4a6-df7dfb0cbdd6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.706312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.707001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.707430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.707565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.709018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.709807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.734369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 409/822] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:13:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:13:59.735389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.759976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef20467-29f7-48a9-90b8-6b0a8799d713 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.811381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.812729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.813312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.813786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.823587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.825794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:13:59.854897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:13:59.856139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:13:59.857725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:13:59.866466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 414/823] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:13:59.872688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.874643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f077db-ff67-4c1e-a8ce-51d2d3d8d071 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:13:59.915026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:13:59.915726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:13:59.916014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:13:59.916296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:13:59.917161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:13:59.917478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:13:59.919011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:13:59.934273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 410/824] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:13:59.940858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:13:59.970531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.039667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.064368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d27e3-89c2-410b-a477-4a92494cd1c4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.068397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598c134-0430-4db6-ba2f-8c033d080f96 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.110327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.111050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.111844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.111844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.112622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:00.112754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.114233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9b3f9-bfd4-4694-9cc4-0250936264eb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.114486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.115414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.115701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.115970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/5407b0778630459ba6a046d63d69cd02/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.138792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=5407b0778630459ba6a046d63d69cd02, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.139464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 415/825] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:13:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:00.140815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.141537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fedb03-da8e-4b1b-ac80-f06161570793 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.141537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fedb03-da8e-4b1b-ac80-f06161570793 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.141537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fedb03-da8e-4b1b-ac80-f06161570793 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.143128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 416/826] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:00.148778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.149067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.149351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.169023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 411/827] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:13:59 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/5407b0778630459ba6a046d63d69cd02/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:14:00.171546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.172046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:00.173438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:00.268759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 417/828] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:00.288962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.304694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.345198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea5f2c-d8a8-4092-b705-498c625cd230 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.378377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.392131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.393213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.393672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.394142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.416286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79742b0b-56bf-4872-9a6c-7f3be36d3465 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.465022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.466243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.466718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.467738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.469963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.471907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214927f-d0b3-4096-ba4d-60a850aa3952 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.480785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e08bca1f-da40-46cc-8092-15bf82936d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:00.492604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:00.493678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:00.494524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:00.496099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 418/829] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:00.499440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b509-b2f8-4d7c-ba2e-298778ad363d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.499719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b509-b2f8-4d7c-ba2e-298778ad363d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.499995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6b509-b2f8-4d7c-ba2e-298778ad363d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.501616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 419/830] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:00.504294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6b509-b2f8-4d7c-ba2e-298778ad363d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.553261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.597277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 412/831] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:00.597277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.597277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.597277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.610922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:00.612153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:00.633230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6b509-b2f8-4d7c-ba2e-298778ad363d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.665630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.675078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.675986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.676374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.676752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.677796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:00.678205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.680160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.694492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 420/832] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:00.695925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.710517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 413/833] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:00.711387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.832118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf2d5d-d14e-446b-ab4d-68859652a89c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.835615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:00.842872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e1660-3fe9-406d-ad1a-9d7282198465 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:00.881187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.882223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.882552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.882833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.887159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:00.887983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:00.888275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:00.888650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:00.896196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.897725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.900241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:00.901727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:00.916365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:00.917335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 421/834] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:00.923868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:00.924202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:00.925041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 414/835] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:00.925902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:00.975714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.031341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dccad-3696-4306-bd3f-6261482d9017 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.035213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0aecb-34d8-444a-bf3e-83ea2a9cc476 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.067310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.068607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.068897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.069314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.069889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.071208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.074071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.074742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.075036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.075314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.076212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.076788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.077819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:01.083094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:01.084018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:01.092468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 422/836] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:01.093244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.098551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 415/837] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.103534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.111894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.200893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe632b-216f-4299-907a-26a3c228ddd7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.210590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.212291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83e670-60a8-4702-82ec-8577154c7f7b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.238526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.239198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.239482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.239757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.240523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.240832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.242231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.250116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.250784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.251070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.251345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.252061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.252393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.253773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.257057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 423/838] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.258194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.264782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 416/839] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.265915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.345745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.373727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24eb7f-eb4e-4c01-bb8d-aec4f301e228 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.379583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292184c-2dd4-4232-a7e8-7f2d1517fd96 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.410692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.411496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.411777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.412055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.415184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.416049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.416419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.416753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.419180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.420894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.424457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.425955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:01.436295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:01.437379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:01.438518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 424/840] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:01.439086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:01.445774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:01.446901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:14:01.448028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:01.454683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 417/841] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:01.454683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.469165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.542602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af4f0d-b86c-47ee-b4c0-6fd0fc75c78f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.560261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452c813-482c-43b0-bf8c-f67ddb4fbb58 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.578520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.579381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.579880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.580297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.581194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.581636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.583153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.588084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 425/842] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.589034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.600978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.607399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.608485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.608952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.609205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.610018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.610333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.611865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.629604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 418/843] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.631298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.722085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:01.736068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bef34-e652-4759-ace6-135f26211e4e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.774976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.776142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.776660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd65038-dad3-4a38-902d-089fc46db017 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.776797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.777274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.778367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.778763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.780444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.801869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 426/844] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.807440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:01.824741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.825423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.825707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.826046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:01.826909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:01.827227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:01.829292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:01.838313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 419/845] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:01.940436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b5c91-3569-4c24-af3a-2c51952d0cff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:01.979825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:01.980458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:01.980755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:01.981027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.002375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.059306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.061117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b10e46-e8f4-46b1-9106-bb9ae82add49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.173741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 427/846] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:02.179500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.239772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.292014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f06e9-2ee3-4931-b437-a7cb246858fd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.294490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605bb1-2139-447e-882b-a6babf0e4d1b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.294928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605bb1-2139-447e-882b-a6babf0e4d1b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.295387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee605bb1-2139-447e-882b-a6babf0e4d1b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.297248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 428/847] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:02.305029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.305346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.305597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.325629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:02.326915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:02.329117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.329978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.330263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.330538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.342315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.343783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:02.361855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:02.363486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:02.364391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:02.364391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:02.364391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:02.364391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:02.364391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 420/848] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:02.369406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.395902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.424368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 429/849] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:02.449170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.520946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e483a8e-6282-4094-b929-1a702cde42d2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.533931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.565712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db68934-a27b-4d56-bcc5-afbe182bb973 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.571851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.572851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.573339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.573844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.575307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:02.575999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.578949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.599379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 421/850] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:02.601182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.609911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.610730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.611008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.611278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.622147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:02.630020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:02.630962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:14:02.631825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:02.636378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 430/851] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:02.637554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.684598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.726925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae608b63-0811-43dc-889c-5f26d622ced5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.746310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ab8f7-1b76-42a7-a8c2-dfeb4c2b0748 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.766914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.768185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.768706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.769209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.777845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.779490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.783135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.784089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.784681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.785093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.785985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:02.786447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.788030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:02.795877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:02.796951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:02.798007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:02.804229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 422/853] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:02.804437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 431/853] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:02.805575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.806995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:02.838997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.934315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558c9b1-3d8c-4ead-baa9-78498503848c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.948903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094d2d4-3a31-4b36-a246-bbf829bdbd3d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:02.950595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:02.972101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.972733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.973019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.973298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.974100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:02.974403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:02.975990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:02.988471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:02.994061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 423/854] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:02.994210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:02.994210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:02.994210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:02.996119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:03.008146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:03.010582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:03.033434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:03.034352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:03.035230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:03.035230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:03.035230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:03.035230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:03.047111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 432/855] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:03.049377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:03.096174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:03.112863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aaacb-900f-4e82-9088-1a7d6df89234 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:03.163315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:03.164349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.164811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.165261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.166279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=47acb0dcc4164628a2442c7dd26d478b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:03.166800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:03.168645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:03.175990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 424/856] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:02 2022] GET /identity/v3/role_assignments?user.id=47acb0dcc4164628a2442c7dd26d478b&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:03.179064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4634c36-ce74-4bd8-8bd5-592e63702bc6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:03.225344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:03.226463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.226946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.227441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.228441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:03.229003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:03.230656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:03.245983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 433/857] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:03.251569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:03.373948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b34901-1ee7-4108-8545-dc48eb27d814 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:03.413680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:03.414986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.415487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.415958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.424670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:03.426477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:03.442848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:03.444182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:03.445139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:03.447800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 425/858] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:14:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:03.453489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:03.472564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:03.561188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b51a61-8468-4a11-8ff8-c8dcbbf46ef2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:03.721132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:03.721734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.722017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.722286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.723134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:03.723440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:03.724925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24a3ef-3ee7-448f-9614-423a25cd9029 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:03.748428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 434/859] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:14:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:03.750831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:03.768675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:03.888567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309d3f9-9d45-414d-9d65-4b326d8908f9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:03.926545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:03.927203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.927483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.927752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.949128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d24fa-2d5e-4bf9-8b27-eb8c988360f0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.949658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d24fa-2d5e-4bf9-8b27-eb8c988360f0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.949658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d24fa-2d5e-4bf9-8b27-eb8c988360f0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.951669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 435/860] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:03.957407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:03.957700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:03.957987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:03.975449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:03.976622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:04.006194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.008045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd1f29-bdd4-4015-90b2-0a3dae145c76 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.023727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.052641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 436/861] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:04.077414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.079637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.128350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 426/862] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:04.128350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b803c2-0142-465f-889d-abd525b7bba3 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.128350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b803c2-0142-465f-889d-abd525b7bba3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.128350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b803c2-0142-465f-889d-abd525b7bba3 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.131172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 427/863] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:04.132183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b803c2-0142-465f-889d-abd525b7bba3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.196585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac72fd8-688b-4e56-843f-3e0e11d76736 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.235882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.236849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.237229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.237674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.238829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.241060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb2e2-367b-45b7-9b7e-e652fd58aed2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.250203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b803c2-0142-465f-889d-abd525b7bba3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.268601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 437/864] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:04 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:04.268601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.268601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.268601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.272704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.302748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:04.305587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:04.321090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.322232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.322679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.323164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.341418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.344254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:04.376228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:04.378180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:04.379593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:04.379593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:04.379593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:04.379593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:04.379593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 428/865] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:04.390008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.420516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.427401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 438/866] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:04.448167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.525481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431bbcb-0aea-48d4-bb4c-3c8039632b4e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.567352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.568945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.569358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.569640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.570387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:04.570695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.571261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9166f3-b93f-4e21-b5aa-f3276109d518 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.572179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.589483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 429/867] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:04.597292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.620981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.622203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.622703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.623560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.624502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.626188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.626333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:04.640452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:04.641445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:04.642272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:04.645241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 439/868] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:04.651820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.722241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.726056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877df6-f9c0-4a3b-92c0-63c9ee9dbc76 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.768550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.769407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.769639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.769915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.777144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.778534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.779317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918437-6374-46df-985c-ab1c2fe6234b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:04.792978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:04.793925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:04.795204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:04.799145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 430/869] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:04.800022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.821643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:04.822785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:04.823362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:04.823848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:04.824873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:04.825440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:04.827321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:04.833757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 440/870] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:04.842306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:04.924700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:04.955243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e34571-1f9d-488a-9877-a5754a206886 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:04.979100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce8cd4-24e6-4f01-a176-27f670d4a902 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:05.001239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:05.002402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:05.003241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.003791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.004360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.005549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:05.005757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:05.007744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:05.019774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:05.020916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.021431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.021909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.022929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:05.028095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:05.028236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 431/871] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:05.029609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:05.030074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a2d26-9174-449e-be80-01e811062b2a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:05.035902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 441/872] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:05.146029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffc346-ac49-46dd-97a6-27e8f0efa40f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:05.184885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:05.186021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.186449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.186875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.187867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=5407b0778630459ba6a046d63d69cd02) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:05.188344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:05.190143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c660d0-769e-4c0a-aa2f-bb5c4724901e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:05.198067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 432/873] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:05 2022] GET /identity/v3/role_assignments?user.id=5407b0778630459ba6a046d63d69cd02&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:05.454919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82493-5c6e-404a-a0bf-df7d08f6d0b6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.455291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82493-5c6e-404a-a0bf-df7d08f6d0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.455577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc82493-5c6e-404a-a0bf-df7d08f6d0b6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.457287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 442/874] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:05.463413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.463913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.464001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.470813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:05.485394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:05.485394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:05.558243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 433/875] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:05 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:05.586721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc82493-5c6e-404a-a0bf-df7d08f6d0b6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:05.695353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc82493-5c6e-404a-a0bf-df7d08f6d0b6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:05.730478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:05.731226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.731514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.731793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.732710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:05.734210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:05.747011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:05.748681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:05.749542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 443/876] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:05.755054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:05.861828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e4c4f-77ba-4b1f-8e34-b7a9312b5c71 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:05.900687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:05.901371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:05.901659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:05.901942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:05.902656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:05.902968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:05.904486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551882e-2bcf-47de-9a29-5052f29af555 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:05.910041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 434/877] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:05.910599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:05.916406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:05.972737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:06.033716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893fa85-b510-4eb8-854e-a6e341ddca64 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:06.070220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:06.071132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:06.071421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:06.071699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:06.072471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:06.072973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:06.074555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0fda9-6562-49c5-b80e-f1987376190e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:06.080995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 444/878] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:06.517849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8af711-85ac-4fbf-abda-f50842164351 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:06.518855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8af711-85ac-4fbf-abda-f50842164351 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:06.519339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8af711-85ac-4fbf-abda-f50842164351 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:06.521235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 435/879] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:06.527005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:06.527548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:06.528018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:06.549407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:06.551376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:06.628088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 445/880] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:06 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:06.654786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8af711-85ac-4fbf-abda-f50842164351 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:06.770404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8af711-85ac-4fbf-abda-f50842164351 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:06.810005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:06.811378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:06.811378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:06.811711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:06.812374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:06.813834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:06.830268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:06.831328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:06.832159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 436/881] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:06.839802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:06.898016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:06.971497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097cc25-780e-4c3c-92da-3bd6d537f928 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.009014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.009701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.009984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.010263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.010985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:07.011294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:07.012728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ff3fe-77da-4d57-ab6e-34a375dd8f94 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:07.018131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 446/882] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:07.023920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.094141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:07.145180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d725-a39e-4c58-844c-6a6cc8fc2253 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.194680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4a923-0fe9-474c-8068-2fcbe7fdcf60 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.195499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4a923-0fe9-474c-8068-2fcbe7fdcf60 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.195499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4a923-0fe9-474c-8068-2fcbe7fdcf60 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.197933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 447/883] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:07.202624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.202949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.203194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.213976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.215636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.216465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.218245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.219923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:07.220717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:07.221066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:07.221914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:07.224222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:07.235715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 437/884] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:07.245658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.304661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 448/885] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:07.329720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.382184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:07.389744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72cbc1-18fd-4eb6-90d0-6026099df2d0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.417541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:07.427112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.427786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.428069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.428367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.429204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] RBAC: Adding query filter params (interface=public, service_id=c5ce76d0c58b468dac0734ddfc4f2f1f, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:07.429519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:07.430983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd2144-1bba-4fad-aebc-0dc93d527635 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:07.440361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 438/886] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:14:07 2022] GET /identity/v3/endpoints?service_id=c5ce76d0c58b468dac0734ddfc4f2f1f&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:07.440693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c5a8-6c1e-433e-a15d-6cea0858a8ff None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.440985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c5a8-6c1e-433e-a15d-6cea0858a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.441259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049c5a8-6c1e-433e-a15d-6cea0858a8ff None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.442754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 439/887] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:07.447926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c5a8-6c1e-433e-a15d-6cea0858a8ff None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.461554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8e04e-f6cd-4a50-b13a-3999de693e08 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.495834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.496545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.496833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.497120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.497918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:07.499310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25b1d8-bd9c-45a2-ae24-f50e0b24f7f5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:07.513339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 449/888] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:07 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:07.515019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.515303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.515575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.539203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:07.540480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:07.603197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049c5a8-6c1e-433e-a15d-6cea0858a8ff None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.632989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 450/889] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:07.649577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.650329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.650616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.650953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.651706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:07.653237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:07.658619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.660990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 440/890] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:07 2022] GET /identity/v3/services => generated 1268 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:14:07.693182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:07.781389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895a1e-e237-4bb2-8d9f-383aa0d44be9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:07.821106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:07.821955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:07.822241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:07.822517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:07.834881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:07.843306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:07.844263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:07.845376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 451/891] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:07.852701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:07.917621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:07.971318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80428c91-ca5b-44af-ae3d-a20598d059f3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.006601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.007289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.007575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.007853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.008660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:08.009011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.010477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.020761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 441/892] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:08.026333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.118886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:08.135017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadd5b-7a22-4988-a7a5-6231490866c9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.171224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.172033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.172330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.172618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.184234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.185833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fcb4db66-b8cf-49e0-8723-3ed1fd64e950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:08.203961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:08.205374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:08.206271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:08.206271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:08.206271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:08.206271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:08.206271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 452/893] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:08.213885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.318752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431235a9-8cd6-4084-a551-4f59f9aac2d9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.331761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e98d-bf28-4937-898a-fa4e25254efd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.334460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e98d-bf28-4937-898a-fa4e25254efd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.334460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291e98d-bf28-4937-898a-fa4e25254efd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.334673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:08.335080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 453/894] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:08.341835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.342403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.342878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.359787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.360540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.360826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.361101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.361839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:08.362152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.362831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:08.363547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.364498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:08.380494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 442/895] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:08.385835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.443572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 454/896] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:08.467124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.503377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81211977-11ab-49f8-a60b-7dd38577e962 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.534873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:08.543006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.544032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.544722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.545445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.554157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.555971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.570947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8118e4d-25fe-4cd2-ad72-b2b49c0175b1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:08.576111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:08.577075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:08.578012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:08.582649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 443/897] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:14:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:08.583632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.617087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.618473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:08.619778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.620276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.620850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.621886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.623911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.641531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 455/898] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:08 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:14:08.705324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2fac-83ec-4e9b-8419-225ab40db767 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.754413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.755177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.755465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.755741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:08.756696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:08.757062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:08.758746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:08.781282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 444/899] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:14:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:08.786657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:08.911178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb277d-28ae-4e02-866e-550bbbcd5f60 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:08.939259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:08.948305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:08.949176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:08.949629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:08.950293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.022535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:09.024383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:09.146830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 456/900] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:09.153238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:09.251400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:09.267398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e00e52-e345-49f8-9389-e156e4e181d0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:09.306620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:09.308040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.308584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.309084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.329072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:09.331933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:09.366140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:09.367104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:09.367982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:09.367982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:09.367982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:09.367982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:09.369099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 445/901] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:09.374961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:09.406327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:09.491260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74cb7-c10e-4bb9-bd7e-31578faa3afd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:09.534155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:09.535063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.535484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.535890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.537056 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:09.537531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:09.539742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:09.558599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 457/902] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:09.564816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:09.689914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131e82-52fb-4d21-ace4-a3612efd2680 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:09.730484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:09.731365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.731646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.731927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.740212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:09.741776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:09.743608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c68691e3-03c5-45de-ae4c-8865f30612de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:09.757328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:09.758403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:09.759274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 446/903] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:09.766707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:09.873525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:09.891880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2810c85-1e90-4a88-bff0-c2cf2d3f9611 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:09.920258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad44ba-9f5a-4d3b-8ec7-3d664e30fffc None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.920568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad44ba-9f5a-4d3b-8ec7-3d664e30fffc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.920919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad44ba-9f5a-4d3b-8ec7-3d664e30fffc None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.922593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 447/904] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:09.927956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52e8b-b3bb-4d48-83b5-312c805de670 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.928312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52e8b-b3bb-4d48-83b5-312c805de670 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.928546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52e8b-b3bb-4d48-83b5-312c805de670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.946474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a5e52e8b-b3bb-4d48-83b5-312c805de670 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:09.947897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a5e52e8b-b3bb-4d48-83b5-312c805de670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:09.952149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:09.953372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.953950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.954445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.955938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:09.956579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:09.959608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7adedb-10ec-4d93-8b0d-f5afd28ed82b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:09.984356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 458/905] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:09.984786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ad49-c24b-44e7-9c51-31f67b922434 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:09.985212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ad49-c24b-44e7-9c51-31f67b922434 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:09.985648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ad49-c24b-44e7-9c51-31f67b922434 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:09.988038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 459/906] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:09.989366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4ad49-c24b-44e7-9c51-31f67b922434 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.032474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 448/907] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:10.034754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.035290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.035708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.055787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:10.057065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:10.133626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 449/908] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:10.134297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.165213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4ad49-c24b-44e7-9c51-31f67b922434 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.205536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.206223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.206487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.206761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.207590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] RBAC: Adding query filter params (role.id=47453a1db4ea47d8ba94526092885f24, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=5407b0778630459ba6a046d63d69cd02) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:10.207910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.209356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.209478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.215412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 460/909] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:09 2022] GET /identity/v3/role_assignments?user.id=5407b0778630459ba6a046d63d69cd02&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=47453a1db4ea47d8ba94526092885f24 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:10.216167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.241911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.279330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4349427-6723-4f40-9226-c8179c8d11ac None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.321962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.323279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.323737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.324184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.325209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.327733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:10.341095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:10.342761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:14:10.343867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:10.344772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 450/910] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:10.353954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.370971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c6530-6138-4195-80f7-7e45f6713292 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.445690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.447874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.448335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.448858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.450148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.452549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:10.471169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:10.472362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:10.473373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:10.480377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbb8f-3aa6-4ca2-8a97-5c1e9fc0980d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.480885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 461/911] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:10.483337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.509217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.523734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.524852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.525349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.525885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.526804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:10.527324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.529029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.541086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 451/912] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:10.542703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.595881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.622297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12565eb-0ed3-4e15-ba79-95e3cf3e0fd3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.662758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.663910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.664444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.664915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.665870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:10.666433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.668085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.674711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 462/913] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:10.680557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.694641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f3fc-80e4-4a08-9518-3507a06c19aa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.744565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.745198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.745485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.745766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.746992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:10.747311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.749583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.760382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 452/914] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:10.763522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.804877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0548a72-f603-49ac-a508-71bab155e2bc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.834341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.841529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.845634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.845634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.845634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.845634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:10.845634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.848016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.858820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 463/915] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:10.862536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:10.892712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:10.908599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8614c2e-c823-4315-bbbc-bd3ac5982d0a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.944045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:10.944759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.945050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.945331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.946087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:10.947533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f43f51-a9a2-4b93-865b-016abaa3e65d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:10.976347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 453/916] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:10.976347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640750c-73aa-4bda-8b40-745a16dbd80e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.976347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640750c-73aa-4bda-8b40-745a16dbd80e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.976347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640750c-73aa-4bda-8b40-745a16dbd80e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.976347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 454/917] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:10.977564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:10.977863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:10.978154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:10.998063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12a0bb-6c36-47fa-9065-d70c616cf615 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:10.998838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:11.000138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:11.052588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.053274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.053592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.053832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.054673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] RBAC: Adding query filter params (interface=public, service_id=7cc7a952dcdd4cbe96e169100f2199e3, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:11.055629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.057158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.070088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 464/918] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:14:10 2022] GET /identity/v3/endpoints?service_id=7cc7a952dcdd4cbe96e169100f2199e3&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:11.072076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.094296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 455/919] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:11.122575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.162181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:11.168269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:11.215161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb8ce3-db30-460b-81e1-8237224baf1a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.256248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb79c01-1a1c-4abb-a1c1-cf6391f47059 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.256766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.257450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.257734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.258013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.258705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.260150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.267532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 465/920] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:14:11.299877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.300771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.301053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.301329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.302010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.303462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:11.317702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:11.319118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:11.319933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 456/921] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:11.325905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.449454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7ca0d-7237-45af-9db6-40a159672b24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.478661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:11.487807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.488526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.488811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.489093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.489811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:11.490117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.491615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.497108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 466/922] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:11.503167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.609989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:11.613230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06eb9a-3c30-46f8-a4f5-17a014b98fe6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.649047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.649769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.650096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.650373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.651086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:11.651397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.652807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.658884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 457/923] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:11.664876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.779669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67338fa5-d1ce-485c-b7a3-246a71a3dee2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.817449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.818859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.819329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.819807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:11.820960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] RBAC: Adding query filter params (interface=public, service_id=25ceb6109c834ddd943607b35c94a094, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:11.821493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:11.823078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f1039-16b3-4bc9-b629-a9ca70a6b291 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:11.829439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 467/924] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/endpoints?service_id=25ceb6109c834ddd943607b35c94a094&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:11.840357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:11.962411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d06414-047d-4f1b-bae8-4c2c6b34010e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:11.997881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:11.999075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:11.999360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:11.999634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:12.000340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:12.001724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28e964-d4c6-4d2c-b573-565b6491a3ee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:12.008178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 458/925] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:11 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:14:12.393211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601c30-9f5c-4df4-af0d-775bdbb87b69 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:12.393869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601c30-9f5c-4df4-af0d-775bdbb87b69 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:12.394347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c601c30-9f5c-4df4-af0d-775bdbb87b69 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:12.396208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 468/926] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:12.401886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83348-af3a-4839-87ec-d06156290288 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:12.402429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83348-af3a-4839-87ec-d06156290288 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:12.402912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83348-af3a-4839-87ec-d06156290288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:12.426813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-39f83348-af3a-4839-87ec-d06156290288 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:12.428420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-39f83348-af3a-4839-87ec-d06156290288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:12.454794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:12.504032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 459/927] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:12.526972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c601c30-9f5c-4df4-af0d-775bdbb87b69 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:12.600719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:12.648563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c601c30-9f5c-4df4-af0d-775bdbb87b69 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:12.684241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:12.685191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:12.685481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:12.685906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:12.698122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:12.706556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:12.707492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:12.708381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 469/928] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:12.714598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f83348-af3a-4839-87ec-d06156290288 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:12.824039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f83348-af3a-4839-87ec-d06156290288 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:12.862113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:12.862899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:12.864422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:12.864422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:12.864422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:12.864666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:12.865967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:12.876096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 460/929] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:12.881771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.001256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76911c-f819-4d11-a56a-b76726f5db9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.019841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.041452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.042269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.042557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.042884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.055853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.057426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6615c84a-ae35-4070-873a-1d16c9370b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:13.076946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:13.077904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:13.078773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:13.078773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:13.078773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:13.078773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.078773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 470/930] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:13.085445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.116777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.176104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884dc37-5c56-4353-8874-96f6dd6433e1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.176445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884dc37-5c56-4353-8874-96f6dd6433e1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.176722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884dc37-5c56-4353-8874-96f6dd6433e1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.179682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 471/931] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:13.184829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.185142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.185519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.203399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:13.204648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:13.205809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837e1fc-b5bc-4985-b4bb-e0d6b35debad None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.255741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.257099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.257382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.257655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.258390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:13.258697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.260117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.273986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 461/932] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:13.283099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.298850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 472/933] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:13.322973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.394673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.421290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c70dd-4dee-41d6-a267-0cfd0a49a33f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.446350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.451223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bac37d-d664-49dd-9021-3e4de347a7f4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.466374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.467265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.467557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.467835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.477595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.479412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.493401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.494127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.494410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.494691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-08a96226-064b-4760-8d7f-0e2930ac4dd8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:13.494884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:13.495808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:13.496867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:13.497653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.497653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.498544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING keystone.server.flask.common [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 15:14:13.500617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 462/934] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:13.501864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8d42-428e-41a2-8481-33b0d067af95 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.502255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8d42-428e-41a2-8481-33b0d067af95 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.502445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd8d42-428e-41a2-8481-33b0d067af95 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.503929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 463/935] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:13.505134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd8d42-428e-41a2-8481-33b0d067af95 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.524469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.sql.core [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:14:13.524469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 15:14:13.524469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [parameters: {'id': '66a258c03a9d4ff587f81a2d905d2bcb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 15:14:13.524469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80213) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f413e7d4-5d78-413e-a7cc-f4722d92f05a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66a258c03a9d4ff587f81a2d905d2bcb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66a258c03a9d4ff587f81a2d905d2bcb. Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:14:13.541594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 15:14:13.543015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application yield resource Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise self.value Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application session.commit() Jun 27 15:14:13.544064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:14:13.545160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:14:13.546209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:14:13.547272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [parameters: {'id': '66a258c03a9d4ff587f81a2d905d2bcb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.548479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:13.549442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 66a258c03a9d4ff587f81a2d905d2bcb. Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 473/936] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jun 27 15:14:13 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.550506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.551202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.565213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:13.567278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:13.640543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd8d42-428e-41a2-8481-33b0d067af95 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.656513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 474/937] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:13.657839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.684884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.685972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.686264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.686548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.687423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:13.687742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.689270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.701280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.711799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 464/938] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:13.712545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.765190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.785963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea9c69-479d-45e1-876d-5dd7d45217e4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.821816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.822730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.823084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.823428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.827864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c6e4e-5400-42da-9be6-8be923f9989d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:13.831155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.832972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:13.847374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:13.848342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:13.849225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 475/939] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:13.849717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.876300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:13.877161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:13.877454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:13.877735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:13.878405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:13.879810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:13.893221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:13.894216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:13.895060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 465/940] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:13.895917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:13.964491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:13.966137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49538099-dd9d-49f3-b4bc-e24319b3df05 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.003015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.003830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.004114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.004418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.007709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7bd2-1360-4602-82f5-f764086ed1cc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.016698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.018196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.021211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:14.038011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:14.038927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:14.039814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:14.039814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:14.039814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:14.039814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:14.041524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 476/941] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:14.042499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.051983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.052705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.053401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.053401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.054153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.054454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.056029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.084412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 466/942] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.084412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.185069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590372ef-3590-4395-9f78-427f0a653a0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.226602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.227297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.227608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.227864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.228644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.228958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.230388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcde1a-4e78-44ca-bb83-c67298562b04 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.231373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e05f4-db91-4fc7-82df-ee33d3131667 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.243683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 477/943] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.243683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f86a1-3b44-40e9-ab8e-4c7879a1f953 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.243683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f86a1-3b44-40e9-ab8e-4c7879a1f953 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.243683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f86a1-3b44-40e9-ab8e-4c7879a1f953 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.243683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 478/944] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:14.244942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f86a1-3b44-40e9-ab8e-4c7879a1f953 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.267038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.280757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.281781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.282059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.282343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.283081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.283395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.285787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed783c5-c774-4868-8dc1-926b933e422c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.299182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 467/945] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.300763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.301058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.301342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.314279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.322844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:14.324060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:14.359719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f86a1-3b44-40e9-ab8e-4c7879a1f953 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.394934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.395167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 468/946] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:14.395701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.396057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.396454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.396835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.397304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.397680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.399163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.411381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 479/947] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.413406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.508477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd122d4-69db-4d37-af2b-178f87562702 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.523137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.525733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f97e87-1fe7-4383-aa5f-39dd4848675d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.546961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.547919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.548170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.548473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.551832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.557553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.559219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.564491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.565160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.565448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.565717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.566463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.568217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:14.574797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:14.575873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:14.577177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:14.579020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 469/948] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:14.580126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.595681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 480/949] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:14:14.596779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.743390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176bd0-39b2-438b-8e8c-d993aadc1ca8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.762486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29531b-5173-42db-9143-a13f74b264c6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.783300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.784134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.784458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.784738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.785406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.786798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:14.801039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:14.802115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:14.803100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:14.804019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.804019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.804019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.804227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 470/950] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:14.804300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.804300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.804300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.809621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.810149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.824421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 481/951] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.831475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:14.852640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.891095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:14.922437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457c187-9752-45ed-84f8-15d0bf04a29f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.958874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:14.959582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:14.959869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:14.960158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:14.960986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:14.961299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:14.962741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:14.967962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd35b3c-b972-49a6-9e7d-c5b55a2a7d32 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:14.968237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 471/952] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:14.977748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:15.006070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:15.007022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:15.007310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:15.007588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/5407b0778630459ba6a046d63d69cd02/roles/47453a1db4ea47d8ba94526092885f24` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:15.010066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=5407b0778630459ba6a046d63d69cd02, role_id=47453a1db4ea47d8ba94526092885f24)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:15.011667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:15.036655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 482/953] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:14:14 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/5407b0778630459ba6a046d63d69cd02/roles/47453a1db4ea47d8ba94526092885f24 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:14:15.098089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:15.101597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a929550-9df4-4834-aeec-1af6292b91ff None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:15.143946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:15.145080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:15.145572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:15.146055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:15.146986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:15.147513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:15.149330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556496b-c686-446f-9097-3b891a74cad4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:15.155920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 472/954] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:15.161610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:15.273638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:15.282656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaa093-bb9a-4d3b-b5a5-9e4ac24892e2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:15.320854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:15.321527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:15.321807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:15.322083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:15.322855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:15.324328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade3305-5ea4-4b98-bd06-be6e7d52477f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:15.346307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 483/955] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:16.290937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bce81-91ce-4389-b441-4644df48cf33 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.291540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bce81-91ce-4389-b441-4644df48cf33 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.291540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bce81-91ce-4389-b441-4644df48cf33 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.293154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 473/956] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:16.298875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.299786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.300273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.319682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:16.321290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:16.401590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 484/957] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:16.427178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bce81-91ce-4389-b441-4644df48cf33 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:16.538825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bce81-91ce-4389-b441-4644df48cf33 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:16.579157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:16.579760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.580041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.580332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.581054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:16.582504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:16.583832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING keystone.server.flask.common [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 15:14:16.599026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.sql.core [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:14:16.599026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 15:14:16.599026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [parameters: {'user_id': 'a3f9f36c96c445be8d680cbc26309fe8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 15:14:16.599026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Jun 27 15:14:16.606745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:14:16.607832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application yield resource Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise self.value Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application session.commit() Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:14:16.608897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:14:16.609802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:14:16.610717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:14:16.612762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a3f9f36c96c445be8d680cbc26309fe8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:16.614767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 15:14:16.620894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:16.623209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 474/958] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jun 27 15:14:16 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:16.628641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:16.710400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:16.734657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e40eb-940a-4920-b30f-40be5a1eeaa1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:16.771860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:16.773103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.773642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.774170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.782363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:16.784225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ff89635e-6ef3-4042-b9a1-728980f79960 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:16.801876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:16.803175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:14:16.804427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:16.806769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 485/959] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:16.813163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:16.853690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:16.877683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a810-0436-49de-931d-bd300644a38d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.878340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a810-0436-49de-931d-bd300644a38d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.878857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68a810-0436-49de-931d-bd300644a38d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.881821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 486/960] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:16.886376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.886945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.887484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.904925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:16.906469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:16.946236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e24a1-eda6-4d5b-aaa0-840634292ebc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:16.980432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 487/961] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:16.988269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:16.989040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:16.989328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:16.989661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:16.990478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:16.990840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:16.992581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.000377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.043310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 475/962] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jun 27 15:14:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:17.084743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.114420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f6493-5b6c-4bfc-b88a-0630f699c945 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.155181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.156462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.157048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.157554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.158517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.160200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d31a6318-f5ef-4ab7-8258-1be0b3d81085 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:17.174262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:17.175395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:17.176265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:17.184042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 488/963] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:17.186809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.194721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9f4a-6733-435f-b832-087b4387c158 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.195411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9f4a-6733-435f-b832-087b4387c158 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.195917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e9f4a-6733-435f-b832-087b4387c158 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.197681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 489/964] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:17.204145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.204826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.205908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.227542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:17.229949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:17.297246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.318139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 490/965] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:17.321764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee4d6-1e65-4bf0-8b87-2b20e5068e6e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.342227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.367005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.367612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.367896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.368183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.368928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:17.369233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.370633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.376948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 476/966] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:17.383409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.457473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159c141-27ca-44c0-b5dd-9973c4266cbe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.493858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.494307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03486b19-6f39-40ae-8c58-db182620c383 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.494709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.495032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.495356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.508476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-35962ea2-608c-4fce-8196-5aef2b8955ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:17.517020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:17.518068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:17.518936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 491/967] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:17.519655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596cbe-57a2-4dba-8159-664bd478f1ef None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.519958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596cbe-57a2-4dba-8159-664bd478f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.520165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596cbe-57a2-4dba-8159-664bd478f1ef None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.521469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 492/968] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:17.524538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596cbe-57a2-4dba-8159-664bd478f1ef None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.533013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.533687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.533970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.534243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.534956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:17.535264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.536673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b55465-2268-49cf-b1cd-33097934353c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.541313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 477/969] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:17.542610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.542920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.543205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.566744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:17.568790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:17.577641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.656457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 478/970] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:17.658681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596cbe-57a2-4dba-8159-664bd478f1ef None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.673708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.688776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.704659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.705383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.705706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.706022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.706822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:17.707157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.708608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.720254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 493/971] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:17.726768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.782685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5914f-a5fc-4990-bc0d-c9f6e3c70501 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.819452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.820415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.820726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.821198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.821749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.823162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.836514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7c42f-d154-436d-94de-cc8feee3bc8a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:17.837803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:17.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:17.839601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 479/972] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:17.845707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.856019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.879440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.880290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.880604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.880917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.892885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:17.894370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:17.913570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:17.914673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:17.915706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:17.915706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:17.915706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:17.915706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:17.919619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 494/973] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:17.921420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:17.958783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fa958-993a-44c3-af9d-0b3ddcb93577 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:17.959652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:17.996242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:17.996951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:17.997235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:17.997635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:17.998375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:17.998760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.000154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.008747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 480/974] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:18.011300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.034933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641be33-115a-41f6-99a0-e533657a6e34 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.073093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.073766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.074048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.074320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.075044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:18.075355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.076881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.086678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.093613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 495/975] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:18.094434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.125396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdd87e-27c5-4f51-814c-3a379821ccbc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.173863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.174417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.174700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.174971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.175721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:18.176027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.177488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.188139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 481/976] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:18.189630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.210589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.215954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afab99-5536-4df2-84eb-e8e283b0d27b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.253111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.253955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.254242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.254995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.263314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.265029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:18.281531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:18.282637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:18.283583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 496/977] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:18.288680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.309289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890141a-80eb-40ad-9a55-2428b30eb148 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.352000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.352763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.353083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.353460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.354434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] RBAC: Adding query filter params (interface=admin, service_id=25ceb6109c834ddd943607b35c94a094, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:18.354806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.356543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.358407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.364937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 482/978] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/endpoints?service_id=25ceb6109c834ddd943607b35c94a094&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:18.371042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.414332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e00c6-ca7b-4d71-b1bc-04d65d25fb13 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.470406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.470992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.471271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.471539 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.472955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:18.473211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.474694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.500664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 497/979] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:18.506354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.510445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6c42c-d26a-4f17-a79f-292c80b76d96 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.525413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.552326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.553072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.553355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.553630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.554343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.555737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.563640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 483/980] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:14:18.629529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4b663-7837-445f-bce8-127cbbad9702 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:18.674158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:18.674764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:18.675051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:18.675336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:18.759813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:18.761384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:18.778274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.877424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 498/981] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:18.883131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:18.949614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:18.990640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7273b0-aba1-47b0-a74f-ab14809d576c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.025349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.026082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.026368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.026643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.038264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.039749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:19.057755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:19.058704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:19.059594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:19.059594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:19.059594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:19.059594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:19.063797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 484/982] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:19.065250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:19.173909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbd9c7-a66a-45ab-b86d-1d0a3e303fb6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.188672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:19.210486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.211165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.211451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.211726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.212488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:19.212797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.214207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.226630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 499/983] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:19.232272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:19.337167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d901d-64b3-449b-997a-77bf6d69b247 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.351325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:19.373589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.374413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.374699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.374981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.382096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.383580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:19.398390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:19.399366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:19.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:19.404532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 485/984] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:19.405941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:19.514200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27857-ebfb-41e2-995a-948ba85ed414 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.548113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.548808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.549087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.549369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.550150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:19.550458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.551904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6fdab-a1be-4c25-9b10-be06e4b332c0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.564961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 500/985] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:19.571237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:19.614306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b152e3-0d50-4fe3-a532-ee26553ff260 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.614485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b152e3-0d50-4fe3-a532-ee26553ff260 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.614766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b152e3-0d50-4fe3-a532-ee26553ff260 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.616271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 501/986] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:19.621529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.621960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.622357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.643806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:19.645099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:19.674203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:19.678427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164131fb-7640-45c2-b68e-11771e8c1ecd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.717161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.717995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.718116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.718362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.719197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] RBAC: Adding query filter params (role.id=47453a1db4ea47d8ba94526092885f24, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=5407b0778630459ba6a046d63d69cd02) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:19.719505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.720986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6b48-e8d0-400f-b08b-9ae4ad7ee19e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.722663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 502/987] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:19.736462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 486/988] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:19 2022] GET /identity/v3/role_assignments?user.id=5407b0778630459ba6a046d63d69cd02&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=47453a1db4ea47d8ba94526092885f24 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:19.746163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:19.867091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a28c06-5bcb-430b-8a1e-c76cc85d3cd9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:19.909294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:19.910050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:19.910351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:19.910636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:19.911409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:19.912864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845bfba8-4d83-4089-89ce-74ec2627828a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:19.929709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 503/989] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:19 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:20.632469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5a9b-6619-4a2d-a921-55a94c3fc053 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:20.633021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5a9b-6619-4a2d-a921-55a94c3fc053 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:20.633021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde5a9b-6619-4a2d-a921-55a94c3fc053 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:20.634696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 487/990] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:20.640412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:20.640903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:20.641227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:20.659386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:20.660624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:20.680655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:20.741806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 504/991] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:20 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:20.771347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5a9b-6619-4a2d-a921-55a94c3fc053 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:20.821486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:20.892696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde5a9b-6619-4a2d-a921-55a94c3fc053 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:20.929828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:20.930664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:20.930948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:20.931222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:20.931889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:20.933331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:20.946577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:20.947688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:14:20.948568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:20.950436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 488/992] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jun 27 15:14:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:20.954408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:21.069021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ddce3-3618-45c0-b00a-89e939994441 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:21.115456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:21.116148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.116455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.116709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:21.117429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:21.117724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:21.119141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:21.132086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 505/993] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:21.133624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:21.242596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdb71a-dc0b-4b0b-92fd-f757e488d24e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:21.244059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:21.288509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:21.289213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.289496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.289813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:21.290542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:21.290881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:21.292347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff70c7-bf6f-4421-99c3-f88e97335262 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:21.298862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 489/994] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jun 27 15:14:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:21.304484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:21.323590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:21.422625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b9de5-d602-472e-b07d-78d00f1f1792 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:21.464379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:21.465077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.465332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.465628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:21.466369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:21.468306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7cb2f-5e86-402c-ad40-7cdbdd712a97 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:21.485807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 506/995] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:21.866599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5be4-1f62-4a9f-8319-e4d2492c27ca None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.867032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5be4-1f62-4a9f-8319-e4d2492c27ca None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.867197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee5be4-1f62-4a9f-8319-e4d2492c27ca None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:21.868625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 490/996] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:21.874486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.874942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.875023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:21.892218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:21.893454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:21.969492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:21.978884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 507/997] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:21.997271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33d438-d274-4522-8087-5a203626280f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:21.998097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33d438-d274-4522-8087-5a203626280f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:21.998454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33d438-d274-4522-8087-5a203626280f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.000006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 491/998] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:22.000526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.005351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.005610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.005899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.023023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:22.024228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:22.096383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 492/999] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:22.107205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa06c4-fe2d-4078-a6ae-fecf2ddbac17 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.117779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.143653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.143871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.144108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.144266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.144961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.146350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:22.158330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:22.159369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:22.161338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 508/1000] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jun 27 15:14:21 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:22.165564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.222444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:22.229558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da7303-8791-441c-9f36-8010adaa8372 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.230350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:22.267271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.268080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.268502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.268787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.269443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.270814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.272824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b548870-b450-44fe-8f39-754fbe68497f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:22.282979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:22.284205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:22.285089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 493/1001] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:22.290541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.311682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.312772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.313200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.313614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.314470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] RBAC: Adding query filter params (name=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:22.314936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.316491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.323622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 509/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:22.329238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.401146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c945478-3333-4d02-9222-8f042fd43c40 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.436387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a88b-0bc2-469d-9f33-f65cb532a509 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.438226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.438928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.439273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.439586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.440355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:22.440679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.442117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.448311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 494/1003] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:22.454209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.472241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:22.481429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.482312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.482599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.482993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.483814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:22.484614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:22.484828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.486238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.496362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 510/1004] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:22.571898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328e622-5bbd-4374-9acf-d9e57f679c9d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.609981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.610679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.610965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.611243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.611971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:22.612372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.613882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.619992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 495/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:22.626675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.734160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e0a5-1998-4d7f-8e60-f22776a77701 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.767250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.767919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.768275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.768501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.769282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] RBAC: Adding query filter params (interface=public, service_id=65526ed8a2f9443396e9c975b10af253, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:22.769593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.770986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626686-0199-4fc2-9e25-1171477225a9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.780897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 511/1006] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/endpoints?service_id=65526ed8a2f9443396e9c975b10af253&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:22.782509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:22.881710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:22.896571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd03218-a5b6-4589-861f-0e0d735573a7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:22.931338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:22.932013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:22.932294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:22.932615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:22.933300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:22.934737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67259bf-0348-4b1a-a1ca-938370561180 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:22.948421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 496/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:22 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:14:23.569248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e182-0818-4aaa-8293-30fa895ff51f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:23.569855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e182-0818-4aaa-8293-30fa895ff51f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:23.569855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e182-0818-4aaa-8293-30fa895ff51f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:23.571380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 512/1008] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:23.577411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:23.577751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:23.577983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:23.595487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:23.596730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:23.678636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 497/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:23.702423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0e182-0818-4aaa-8293-30fa895ff51f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:23.771615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:23.847899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0e182-0818-4aaa-8293-30fa895ff51f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:23.886556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:23.887367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:23.887643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:23.887950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:23.890038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:23.891589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac71acd-65a0-473d-817e-02f41a8491c4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:23.898336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 513/1010] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:23.899605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:23.990898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:24.013741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5482-c10c-4904-be82-c271d6181100 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:24.050837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:24.051520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:24.051807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:24.052089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:24.052842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:24.054309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef640c2-46ad-462a-a8fe-393496d5f1f9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:24.092372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 498/1011] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:23 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:24.587374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53319-7989-4c81-a9ae-b3c898cc9c35 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:24.587943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53319-7989-4c81-a9ae-b3c898cc9c35 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:24.587943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53319-7989-4c81-a9ae-b3c898cc9c35 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:24.589385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 514/1012] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:24.594891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808040-3fa5-4043-bbd2-29d134039121 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:24.595215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808040-3fa5-4043-bbd2-29d134039121 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:24.595445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808040-3fa5-4043-bbd2-29d134039121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:24.613198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0f808040-3fa5-4043-bbd2-29d134039121 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:24.614410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0f808040-3fa5-4043-bbd2-29d134039121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:24.690873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 499/1013] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:24 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:24.714003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53319-7989-4c81-a9ae-b3c898cc9c35 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:24.873436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53319-7989-4c81-a9ae-b3c898cc9c35 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:24.901596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:24.915348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:24.916029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:24.916310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:24.916611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:24.917442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:24.918829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551b395-71c6-4246-b7c0-0c6210cfecb4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:24.944350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 515/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:24 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:25.026915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee7f2-cecb-405c-a72b-eeb8f3a6f98d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.027901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee7f2-cecb-405c-a72b-eeb8f3a6f98d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.027901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ee7f2-cecb-405c-a72b-eeb8f3a6f98d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.029213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 500/1015] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:25.034871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.035513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.035513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.054276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:25.055560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:25.142721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 516/1016] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:25 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:25.166800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ee7f2-cecb-405c-a72b-eeb8f3a6f98d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:25.243530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:25.295136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ee7f2-cecb-405c-a72b-eeb8f3a6f98d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:25.337182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:25.337992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.338271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.338546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.339208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:25.340610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:25.353994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:25.355186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:25.356115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 501/1017] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:25.362798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:25.471387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78714a-27e1-4ded-98cc-e8995f649384 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:25.482549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:25.509421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:25.510097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.510377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.510651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.511376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:25.511680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:25.513107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:25.519951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 517/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:25.524461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:25.631088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ad833-04c6-4186-b2c3-ffdd645f025d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:25.673206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:25.683182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:25.683871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.684164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.685487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.685487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:25.686175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:25.687028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503971-cdbb-4be1-a8ff-b5790a10d118 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:25.697800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 502/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jun 27 15:14:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:25.698887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:25.802991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fa19d-cfeb-4e95-84c0-d4c34fb02811 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:25.839720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:25.840413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:25.840699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:25.840973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:25.841798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:25.845691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f81292-9a2c-4a2a-8f18-141857ded93d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:25.872429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 518/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:26.244541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc8b62-9dac-4ecf-a643-6f80ea9cb77e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:26.245057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc8b62-9dac-4ecf-a643-6f80ea9cb77e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:26.245057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc8b62-9dac-4ecf-a643-6f80ea9cb77e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:26.246374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 503/1021] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:26.252393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:26.252714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:26.252998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:26.270057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:26.271257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:26.296550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:26.348698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 519/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:26.374139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc8b62-9dac-4ecf-a643-6f80ea9cb77e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:26.481754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc8b62-9dac-4ecf-a643-6f80ea9cb77e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:26.518293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:26.519104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:26.519397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:26.519676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:26.521018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:26.522472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ee5b8-791b-4b61-bb31-27be980e6ac0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:26.529368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 504/1023] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:26.531167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:26.633939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe9eb-fff1-43de-a5ba-d975e469b609 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:26.669573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:26.670777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:26.671243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:26.671697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:26.672632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:26.674305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2f913-acaa-490c-9ec6-1b64679feb77 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:26.698473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 520/1024] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:14:26 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:27.156875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e502f-65e7-46a6-93da-f47f9301ab9b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:27.157080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e502f-65e7-46a6-93da-f47f9301ab9b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:27.157351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e502f-65e7-46a6-93da-f47f9301ab9b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:27.158925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 505/1025] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:27.165348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:27.166043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:27.166043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:27.186412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:27.187905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:27.197389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:27.269576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 521/1026] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:27.292586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e502f-65e7-46a6-93da-f47f9301ab9b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:27.369693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:27.418202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e502f-65e7-46a6-93da-f47f9301ab9b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:27.457877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:27.458726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:27.459007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:27.459282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] PATH_INFO: `/v3/services/image` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:27.459947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:27.461379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:27.475640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:27.476760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:27.478072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:27.479439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 506/1027] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jun 27 15:14:27 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:27.484262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:27.605317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d3c27-9b71-4529-8ed9-d35ce03aee33 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:27.664553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:27.665344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:27.665795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:27.666251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:27.667059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] RBAC: Adding query filter params (name=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:27.667426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:27.668921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:27.680359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 522/1028] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:27.681770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:27.811512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b986e-c786-4727-9a15-122ab52f30df None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:27.863070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:27.886789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:27.887487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:27.887772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:27.888049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:27.890030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:27.890513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:27.892112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f394-385a-4654-9c44-52165af4bdf5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:27.903419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 507/1029] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:27.904913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:27.915385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:28.029694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43918893-665b-43d0-9a52-d52f0b0d167a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.072482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.073182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.073471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.073748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.074660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493524fa-0b68-4778-8c55-4ade9932b134 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.074808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] RBAC: Adding query filter params (interface=public, service_id=388dc8925ce34ac0805ac8cb6bd8ad3b, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:28.075099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493524fa-0b68-4778-8c55-4ade9932b134 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.075216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493524fa-0b68-4778-8c55-4ade9932b134 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.075522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:28.077191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 508/1030] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:28.077469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:28.083351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.084070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.084382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.085893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 523/1031] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:14:27 2022] GET /identity/v3/endpoints?service_id=388dc8925ce34ac0805ac8cb6bd8ad3b&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:28.091605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.109945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:28.111760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:28.171377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:28.200392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 509/1032] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:28.211821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6efdb-71e4-48f5-b6b4-160c034cf7ef None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.223992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.256690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.257634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.257918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.258201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.259130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:28.260413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:28.268344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 524/1033] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:14:28.345477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa40ca9-6ae5-4ba9-bf39-43412740cbb7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.367126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:28.387192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.388083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.388450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.388718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.402467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:28.411026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:28.412084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:28.417623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 510/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:28.419141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.531138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ddbbf-00bc-42a9-b8a7-9aa0fe5bcead None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.568115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.568830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.569111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.569381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.570137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:28.570445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:28.571836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:28.573333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:28.582306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 525/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:28.588058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.691995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c4017-9f7b-44b0-8019-e75a4e26f472 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.727369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.728181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.728513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.728791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.740407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:28.741880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:28.761669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:28.762639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:28.763684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:28.763684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:28.763684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:28.763684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:28.767681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 511/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:28.769133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.882132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2f28-8127-4095-b73d-5ce4f8e68232 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:28.918107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:28.918794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.919047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.919348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.920299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:28.920546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:28.922005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014d3ce-6ec6-4886-82b7-d8f79024fe74 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:28.939265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 526/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:28.940991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:28.948758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0780bf-1034-4319-b2a3-8890391e5cde None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.949016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0780bf-1034-4319-b2a3-8890391e5cde None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.949239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0780bf-1034-4319-b2a3-8890391e5cde None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.950787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 527/1038] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:28.955756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:28.956038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:28.956309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:28.973905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:28.975099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:28.979114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.032532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.051853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b9ae-caa7-41ab-b891-2d2623cdfcdf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.052665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 528/1039] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:29.077401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.087660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.088539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.088790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.089056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.098114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.099957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:29.114677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:29.115663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:29.116652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 512/1040] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:14:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:29.121656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.207711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2022e1-3ef1-4ccb-865d-a02d8976ce0b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.226659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.237752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c74f-744b-408c-8daa-c2858ff9b94c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.254811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.255856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.256247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.256940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.271205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:29.274565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.275250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.275530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.275803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.276676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:29.276982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.278413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:29.278736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:29.280611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:29.281837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 529/1041] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:29.285898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.302261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 513/1042] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:29.307862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.340796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.399397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d714e7-b62d-4e71-8768-965c7d9a5b24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.418014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a6b6-24c2-4f1b-b5e2-05e07bc86d6e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.436925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.437716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.437992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.438266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.439019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:29.439322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.440841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.451071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 530/1043] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:29.456741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.459107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.459712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.459991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.460264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.480927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.534143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.535689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.563969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dd7b6-2c12-4840-9c6e-cba7e3ebb582 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.574992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.606318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.608469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.608982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.609450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] PATH_INFO: `/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.628358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5cf6bf44c00e40c598546b3e4b0e8d37)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.630764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.633901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 531/1044] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:29.639124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 514/1045] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:29.639699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.644961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.704392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.743264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcf4b7-4141-4463-b38a-4d42a0ae28f4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.759170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b2a23-7ae7-4bea-a510-569ae6a1ade1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.785499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.786234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.786529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.786786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.798761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.799869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.800153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.800598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.812189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.813686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.823349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:29.832877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:29.833840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:29.834711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:29.834711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:29.834711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:29.834711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:29.834711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 515/1046] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:29.839714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.870653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.872138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:29.944924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:29.948559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb06c-8cce-45ba-8e94-1ecf8b9329f9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:29.984418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 532/1047] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:29.986991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:29.991053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:29.991738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:29.992023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:29.992300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:29.993062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:29.993427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:29.994765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.007525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 516/1048] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:30.012847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.059956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:30.101488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2226a-11a2-40d0-9282-ee8537b66a81 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.120421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90beac56-8b1a-433f-aa0d-89312b173ed2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.146351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c163-d7ce-4364-a825-03911900108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.147385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c163-d7ce-4364-a825-03911900108a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.147660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c163-d7ce-4364-a825-03911900108a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.147984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841c163-d7ce-4364-a825-03911900108a None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.149252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c163-d7ce-4364-a825-03911900108a None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.150681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841c163-d7ce-4364-a825-03911900108a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.158175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 533/1049] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:29 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:30.159387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c163-d7ce-4364-a825-03911900108a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.163076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.164571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.164571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.164571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.172459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.173930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:30.189487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:30.190627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:30.191562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 517/1050] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:30.197136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.221217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:30.278467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841c163-d7ce-4364-a825-03911900108a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.315176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b34e4e-e5d7-4f4a-9def-c7404d3a46e2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.316874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.317546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.318194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.318468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.319321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf, user.id=5cf6bf44c00e40c598546b3e4b0e8d37) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:30.319615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.321136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a25118-feeb-42d8-961c-8ebd91366e15 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.323558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:30.339623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 534/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/role_assignments?user.id=5cf6bf44c00e40c598546b3e4b0e8d37&scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:30.344151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11372-14db-40d5-8ec1-89745b8bd719 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.344524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11372-14db-40d5-8ec1-89745b8bd719 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.345032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc11372-14db-40d5-8ec1-89745b8bd719 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.347093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 535/1052] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:30.353110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.353485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.353669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.356742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.357499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.357812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.358071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.358928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:30.359243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.360814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.371757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:30.373143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:30.381468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 518/1053] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:30.382764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.464163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 536/1054] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:30.487449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:30.491603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.513228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687d0b-c580-47f6-8797-7e6539bc302c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.551756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.552728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.552728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.552904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.553755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] RBAC: Adding query filter params (role.id=47453a1db4ea47d8ba94526092885f24, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:30.554059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.555519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.568279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 519/1055] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=47453a1db4ea47d8ba94526092885f24 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:30.615492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2402574-07bf-464a-8138-777e46280ae6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.646889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:30.656427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.657255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.657533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.657802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.658468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.659913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:30.672987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:30.674080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:14:30.674949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:30.679534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 537/1056] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:30.681070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.803641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d3b2-fade-427f-a18a-bccdd7f9614a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:30.842280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:30.842990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:30.846239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:30.846239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:30.846239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] RBAC: Adding query filter params (name=image) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:30.846239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:30.846239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:30.856483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 520/1057] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:30.858690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:30.971287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c51b-823e-4dd2-a745-080b19d4a68f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:31.007245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:31.007918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:31.008204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:31.008573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:31.009301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:31.009602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:31.010993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190913-96f3-47e3-ab87-dc8226ebf551 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:31.017371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 538/1058] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:31.023135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:31.036548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:31.141772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b2157-879e-4f72-a61c-6a8cdfe99900 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:31.179322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:31.180008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:31.180288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:31.180594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:31.181357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:31.182742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d68b1-e165-441a-b578-fb00bd0306b2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:31.212359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 521/1059] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:14:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:32.451600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a3808-7630-4be8-9d02-c2ee265c983b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.452177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a3808-7630-4be8-9d02-c2ee265c983b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.452177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a3808-7630-4be8-9d02-c2ee265c983b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.453577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 539/1060] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:32.459371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab75a-3e11-4d04-9ff3-2c83a2f225d9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.459851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab75a-3e11-4d04-9ff3-2c83a2f225d9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.459999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ab75a-3e11-4d04-9ff3-2c83a2f225d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.478080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-eb7ab75a-3e11-4d04-9ff3-2c83a2f225d9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:32.479297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-eb7ab75a-3e11-4d04-9ff3-2c83a2f225d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:32.543338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:32.559547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 522/1061] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:32.582668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a3808-7630-4be8-9d02-c2ee265c983b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:32.642421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b5b3-e6c3-407e-9e56-adb0f4899276 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.642748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b5b3-e6c3-407e-9e56-adb0f4899276 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.642977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b5b3-e6c3-407e-9e56-adb0f4899276 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.644930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 523/1062] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:32.650150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.650420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.650690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.651650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:32.668131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:32.669395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:32.703136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a3808-7630-4be8-9d02-c2ee265c983b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:32.739046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:32.739861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.740141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.740441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.742389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 524/1063] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:32.752712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:32.760431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:32.761473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:32.762332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 540/1064] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:32.768372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:32.771184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:32.887149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702045d1-7375-42c7-994b-b03cbc6f846a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:32.891283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2dde86-483e-42e3-a28a-3873407585bb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:32.909539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:32.909740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:32.927180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:32.927870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.928146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.928451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.929207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:32.929513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:32.930635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:32.930890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:32.931464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:32.931744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:32.932008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:32.946222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 525/1065] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:32.946439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:32.947408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:32.951696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:32.952705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:32.953615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 541/1066] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:32.959111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.064973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c998f9-2dd1-40bd-9489-ec9c18035326 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.082348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde44f-f3db-45b4-8f4d-2a0ae9fddc08 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.104856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.105742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.106025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.106294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] PATH_INFO: `/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.108307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=5cf6bf44c00e40c598546b3e4b0e8d37)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.109785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.116020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 526/1067] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:32 2022] GET /identity/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:33.117120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.124124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.124889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.125169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.125441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.126202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:33.126938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.128221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.137406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 542/1068] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:33.144365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.184199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.190935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.240046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beab040-c0fe-460d-b160-33909c6df5c3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.258114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407e142-9890-463e-a86d-80064e0212e1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.287169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.288055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.288358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.288643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.289839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.291326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.293093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 527/1069] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:33.298990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.307757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.308823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.309241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.309671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.322334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.323800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-276f91b7-469a-4c28-ad64-489a7f303aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:33.341448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:33.342752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:33.343660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:33.343660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:33.343660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:33.343660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:33.346179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 543/1070] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:33.347489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38abb1-0337-4bf3-bf52-4b439624d01b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.347798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38abb1-0337-4bf3-bf52-4b439624d01b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.348078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38abb1-0337-4bf3-bf52-4b439624d01b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.349600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 544/1071] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:33.351962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38abb1-0337-4bf3-bf52-4b439624d01b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.403400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d564c-b37d-4886-a022-7a82d7ea6f6a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.421085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.438562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.439471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.439753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.440031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf/users/5cf6bf44c00e40c598546b3e4b0e8d37/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.442028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf, user_id=5cf6bf44c00e40c598546b3e4b0e8d37, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.443846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a2b295-1640-46bc-b84c-1fc31ee9c66f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.451454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.458979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 528/1072] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:14:33 2022] PUT /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf/users/5cf6bf44c00e40c598546b3e4b0e8d37/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:14:33.460367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.460940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.460940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.464980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38abb1-0337-4bf3-bf52-4b439624d01b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.479678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:33.480988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:33.513236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.514472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.515327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.516044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.517060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:33.517524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.519121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.540074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 545/1073] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:33.541481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.572377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 529/1074] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:33.593467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.666478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb6c6c-c509-4bfe-bf0d-b545530b94f0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.701859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080f27e-7c47-405b-83ae-e9fea443ad6c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.704804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.705608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.705888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.706159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.712071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.714422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.716074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:33.733768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:33.734784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:33.735721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 546/1075] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:33.740905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.745718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.746390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.746668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.746953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.753870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:33.831826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.833281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.869154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc6704-edf9-4cb7-99ab-ec16eee3d3b5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:33.907448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:33.908247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:33.908654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:33.908942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:33.909790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:33.910392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:33.911587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:33.935323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 547/1076] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:33.936521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.956378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 530/1077] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:33.962596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:33.973168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.010113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.047911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f9571-f2a1-427c-8372-440b228e03e8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.073255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2083d-2e8f-4f5d-a151-c25fd98f1196 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.085587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.086314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.086599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.086877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.098468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.100000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.111181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.111997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.112282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.112594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:34.118890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:34.119774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:34.121570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:34.121570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:34.121570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:34.121570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:34.121570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 548/1078] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:34.125100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.126304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:34.126951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:34.149381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:34.150365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:34.157519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:34.157519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:34.157519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:34.157519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:34.157519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 531/1079] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:34.159301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:34.222458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.236468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9ab52-f39d-45f6-9fc9-62f71bd5d6c5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.258445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.277421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a8be-d2be-471a-9fd3-1fcd8ebf581f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.283143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.284176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.284654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.285097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.286290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:34.286788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.289136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.308390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 549/1080] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:34.315536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:34.330880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.331834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.332752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.332752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.333727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:34.334219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.335864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.348996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 532/1081] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:34.449190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b4f90-6afd-4603-abad-af5ef4c27fa8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.500453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.501358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.501578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.501852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.510805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.512420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:34.529032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:34.530308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:34.532059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 550/1082] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:14:34.537271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:34.679694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425020b-48dc-4f0f-9275-9b04fbe22bc1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.708984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.733922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.734697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.735009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.735308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.736116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:34.736446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.738153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb4786-7b6f-4db1-b18f-c1f418a1f9e8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.752986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 533/1083] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:34.761448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:34.816967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:34.885311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314375-0258-4179-ad53-078e795377c5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:34.922418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:34.923324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:34.923608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:34.923882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/47453a1db4ea47d8ba94526092885f24` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:34.926442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=c3f4d2d7923a4a9a8f59968f770949d6, role_id=47453a1db4ea47d8ba94526092885f24)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:34.927923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b336937-7103-49c2-913b-6db9f128f951 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:34.944705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 551/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:14:34 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/c3f4d2d7923a4a9a8f59968f770949d6/roles/47453a1db4ea47d8ba94526092885f24 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:14:35.563743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba710-dbe5-4640-a4e7-f8df7577bec3 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:35.564386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba710-dbe5-4640-a4e7-f8df7577bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:35.564639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba710-dbe5-4640-a4e7-f8df7577bec3 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:35.566819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 534/1085] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:35.574438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:35.575066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:35.575066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:35.599864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:35.601496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:35.692509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 552/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:35.719918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba710-dbe5-4640-a4e7-f8df7577bec3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:35.883557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba710-dbe5-4640-a4e7-f8df7577bec3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:35.933874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:35.934876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:35.935250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:35.935590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:35.948887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:35.956991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:35.958253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:35.959401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:35.960934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 535/1087] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:35.969038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:36.021191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:36.136722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da19ee8-bbd6-4021-b9a6-33d1a08404e0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:36.189180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:36.190352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:36.190807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:36.191342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:36.192574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:36.193232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:36.195636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:36.208838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 553/1088] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:36.218265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:36.273508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:36.339546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6056c6-4aba-45ad-b6a7-4bf995bb8d16 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:36.376140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:36.377103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:36.377430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:36.377737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] PATH_INFO: `/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:36.379048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] RBAC: Authorizing `identity:get_user(user_id=5cf6bf44c00e40c598546b3e4b0e8d37)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:36.380551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:36.382377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 536/1089] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:36 2022] GET /identity/v3/users/5cf6bf44c00e40c598546b3e4b0e8d37 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:36.389930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:36.498526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:36.510391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab36cc-f509-4613-acaf-f2edda04827e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:36.554399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:36.555377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:36.555961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:36.556298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:36.640742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:36.642355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:36.753785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:36.770048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 554/1090] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:36.777578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:36.897685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bc39c-9ece-412d-8d51-228eb544e83c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:36.910322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:36.949626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:36.951351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:36.951990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:36.952335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:36.953881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:36.955513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380fafe-7b82-4728-afd7-364dae6c420a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:36.957662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 537/1091] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:36 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:36.965018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:37.099666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88ab61-9ba6-4aae-b0d0-042637a49bc7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:37.159917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:37.160600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.160913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.161192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:37.162511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf, user.id=5cf6bf44c00e40c598546b3e4b0e8d37) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:37.162983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:37.165286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759ccde-9a4d-4161-896d-c6a83221237d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:37.173089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 555/1092] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:36 2022] GET /identity/v3/role_assignments?user.id=5cf6bf44c00e40c598546b3e4b0e8d37&scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:37.289468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a38a-1468-4805-aeec-edb3709491d2 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.289836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a38a-1468-4805-aeec-edb3709491d2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.290217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7a38a-1468-4805-aeec-edb3709491d2 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:37.291928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 538/1093] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:37.301182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.301496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.301806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:37.324569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:37.326263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:37.401293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:37.426189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 556/1094] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:37.453845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7a38a-1468-4805-aeec-edb3709491d2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:37.586670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7a38a-1468-4805-aeec-edb3709491d2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:37.625837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:37.626684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.626999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.627360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:37.640699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:37.649156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:37.650170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:37.651005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 539/1095] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jun 27 15:14:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:37.657504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:37.775379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e98a19-f37d-4edf-9ebf-53d03fdf67ac None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:37.817608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:37.818667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.818957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.819228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:37.820004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:37.820340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:37.821774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:37.832866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 557/1096] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:14:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:37.838826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:37.884957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:37.961254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955d6b5-f528-4ebc-9b00-d55b070b85e6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:37.998357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:37.999178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:37.999435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:37.999719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:38.010586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:38.012066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:38.030650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:38.031683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:38.032630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:38.032630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:38.032630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:38.032630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:38.032630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 540/1097] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:38.037751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:38.091181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:38.145115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321774e-e6f7-466b-92d8-894b2d83cda4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:38.185575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:38.186269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:38.186549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:38.186826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:38.187577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:38.187885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:38.189352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:38.203476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 558/1098] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:38.209294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:38.304254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:38.317505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d8d80-879f-403e-bf82-2c8c76e863af None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:38.354040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:38.354846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:38.355133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:38.355409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:38.364237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:38.365747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:14:38.380441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:38.381443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:14:38.382363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 541/1099] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jun 27 15:14:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:38.387812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:38.500648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c601d-6ff8-48f8-a6cd-ec8d2b841ffa None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:38.510261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:38.539094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:38.539778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:38.540052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:38.540348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:38.541209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:38.541518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:38.542964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:38.564197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 559/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jun 27 15:14:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:38.571480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:38.680665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02664e77-c38b-4083-a26f-b442bc4e0cfc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:38.715991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:38.717296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:38.717581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:38.717859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:38.736624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:38.787510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:38.788996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:38.913249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 542/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:14:38.919348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.036157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e58019-781b-4e46-ba12-35d609bc34c1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.083049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.083881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.084156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.084467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.096909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.098444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:14:39.119107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:39.120151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:39.121687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:14:39.121687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:14:39.121687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:14:39.121687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:39.121687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 560/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:39.126238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.178994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:39.233105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73c737-2f8a-443d-bd7a-611d5514082a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.270549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.271227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.271515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.271796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.272548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:39.272857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.274260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaaca56-5c1e-466d-82e5-502ee86d71dc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.286669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 543/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:39.292455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.312884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:39.400240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab91b06-1018-4ca5-9fb7-f91569e5d89b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.413639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0c1ce-69e0-4685-a89c-0b46767a78dc None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.414009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0c1ce-69e0-4685-a89c-0b46767a78dc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.414303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0c1ce-69e0-4685-a89c-0b46767a78dc None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.415730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 544/1104] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:14:39.421681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.422061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.422414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.439513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:39.440780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:39.443361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.444176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.444486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.445184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.453295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.454763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:14:39.469293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:39.470254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:39.471430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 561/1105] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:39.476570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.513624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 545/1106] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:39.539303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.558382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:39.581571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:39.581969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592df1d-da5a-4d73-b7ba-d601461e079a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.617814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.618491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.618775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.619052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.619850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:39.620163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.621662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.634776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 562/1107] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:39.641475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.643073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555457cd-4992-4176-9215-9cc8323a3b5f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.679768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.680530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.680814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.681092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.682327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.683806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.685709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 546/1108] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:39.691860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:39.755833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506f7f-c780-4534-896f-17fe51131f8e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.793004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.793707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.793988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.794267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.795527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] RBAC: Adding query filter params (role.id=47453a1db4ea47d8ba94526092885f24, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=c3f4d2d7923a4a9a8f59968f770949d6) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:39.795843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.797345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfcc0d-1ee6-4c53-9cc9-92f4c19a7482 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.803752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 563/1109] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:39 2022] GET /identity/v3/role_assignments?user.id=c3f4d2d7923a4a9a8f59968f770949d6&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=47453a1db4ea47d8ba94526092885f24 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:39.805560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b608743-1750-4014-912a-cb214fd3c5fd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:39.814099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:39.853601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:39.854309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:39.854594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:39.854966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:39.855786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:39.858529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c64be-0d03-411a-be96-1ecfbc707f14 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:39.886063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 547/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:14:39 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:14:42.120471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea03a-7ce4-4460-bb0d-49995e3a7077 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:42.121220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea03a-7ce4-4460-bb0d-49995e3a7077 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:42.121220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ea03a-7ce4-4460-bb0d-49995e3a7077 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:42.123173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 564/1111] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:42.129796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:42.130536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:42.130536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:42.155236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:42.156640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:42.260603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 548/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:42.285049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ea03a-7ce4-4460-bb0d-49995e3a7077 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:42.288388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:42.410786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ea03a-7ce4-4460-bb0d-49995e3a7077 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:42.454718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:42.455742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:42.456026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:42.456370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] PATH_INFO: `/v3/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:42.458030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] RBAC: Authorizing `identity:get_role(role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:42.459446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:42.461108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 565/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:42 2022] GET /identity/v3/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:42.467308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:42.571548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:42.582078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372ddc-0bef-4ef4-a0b0-bf56885029ac None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:42.620056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:42.621010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:42.621289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:42.621571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] PATH_INFO: `/v3/users/3bdf8ba38a9b4714936d1d01671a43b3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:42.637368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] RBAC: Authorizing `identity:get_user(user_id=3bdf8ba38a9b4714936d1d01671a43b3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:42.638938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:42.640798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 549/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:42 2022] GET /identity/v3/users/3bdf8ba38a9b4714936d1d01671a43b3 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:42.647147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:42.727006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:42.761380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5c1f0-0f89-4612-902d-067e92dc2597 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:42.797708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:42.798361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:42.798638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:42.798907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:42.871149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:42.872702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:42.942604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:42.976832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 566/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:42.983188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:43.087680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da575e3-e16e-4bf5-97f5-68cf0d434093 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:43.126344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:43.127219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:43.127499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:43.127775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:43.129019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:43.130460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c2a1a-736b-41fa-be0a-b3f026ebf314 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:43.132180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 550/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:42 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:14:43.138685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:43.242447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78dda9-18e9-400c-b118-26c46d92ce84 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:43.277844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:43.278515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:43.278821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:43.279121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:43.279949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf, user.id=3bdf8ba38a9b4714936d1d01671a43b3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:43.280263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:43.281741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c6841-1709-45a9-b139-0853f55bd367 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:43.287609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 567/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:43 2022] GET /identity/v3/role_assignments?user.id=3bdf8ba38a9b4714936d1d01671a43b3&scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:45.284398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5786c3-3afb-4c22-bb6c-4bd1b515cc74 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:45.284980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5786c3-3afb-4c22-bb6c-4bd1b515cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:45.284980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5786c3-3afb-4c22-bb6c-4bd1b515cc74 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:45.286365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 551/1118] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:45.292002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:45.292459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:45.292545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:45.310586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:45.311807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:45.342292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:45.389960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 568/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:45.413263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5786c3-3afb-4c22-bb6c-4bd1b515cc74 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:45.434786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:45.531325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5786c3-3afb-4c22-bb6c-4bd1b515cc74 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:45.569655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:45.570441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:45.570726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:45.571002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] PATH_INFO: `/v3/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:45.572596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] RBAC: Authorizing `identity:get_role(role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:45.573990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:45.575438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 552/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:45 2022] GET /identity/v3/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:45.581977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:45.689810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f91d0-83d7-4dbe-9a99-a6fd94a58b4d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:45.727393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:45.728279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:45.728591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:45.728878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] PATH_INFO: `/v3/users/3bdf8ba38a9b4714936d1d01671a43b3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:45.730131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] RBAC: Authorizing `identity:get_user(user_id=3bdf8ba38a9b4714936d1d01671a43b3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:45.731586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:45.733398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 569/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:45 2022] GET /identity/v3/users/3bdf8ba38a9b4714936d1d01671a43b3 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:45.739612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:45.829663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:45.848112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c34b7c-fb76-4894-834e-fbaad7d56d7a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:45.891469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:45.892434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:45.892715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:45.892990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:45.894215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:45.895798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88b54f-2ff4-4b1c-b430-027bab90ea9e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:45.897741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 553/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:45 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:45.904631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:45.938969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:46.014612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90486c-b882-41ce-8c9f-5ffb00d94404 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:46.049747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:46.050643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:46.050926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:46.051201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf/users/3bdf8ba38a9b4714936d1d01671a43b3/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:46.053602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf, user_id=3bdf8ba38a9b4714936d1d01671a43b3, role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:46.055071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d6cd88-3c10-483a-964f-ff3b77eb5abf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:46.070103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 570/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:14:45 2022] PUT /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf/users/3bdf8ba38a9b4714936d1d01671a43b3/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:14:48.112537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a809-931b-417e-baf1-1af240b1202b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:48.113156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a809-931b-417e-baf1-1af240b1202b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:48.113156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a809-931b-417e-baf1-1af240b1202b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:48.114532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 554/1124] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:48.119964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:48.120454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:48.120555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:48.139661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:48.140908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:48.212574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 571/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:14:48.236973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90a809-931b-417e-baf1-1af240b1202b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:48.344822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90a809-931b-417e-baf1-1af240b1202b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:48.381067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:48.381857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:48.382136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:48.382409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] PATH_INFO: `/v3/roles/6b045d9da54b4f4c8474e262e7326a52` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:48.383995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] RBAC: Authorizing `identity:get_role(role_id=6b045d9da54b4f4c8474e262e7326a52)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:48.385429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:48.386838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 555/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:48 2022] GET /identity/v3/roles/6b045d9da54b4f4c8474e262e7326a52 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:48.393008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:48.418843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:48.498863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf114fd-9941-4ba3-9fd8-7451bf3828bb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:48.536644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:48.537567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:48.537847 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:48.538119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] PATH_INFO: `/v3/users/3bdf8ba38a9b4714936d1d01671a43b3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:48.539448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=3bdf8ba38a9b4714936d1d01671a43b3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:48.540990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:48.542719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 572/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:48 2022] GET /identity/v3/users/3bdf8ba38a9b4714936d1d01671a43b3 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:48.549897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:48.577687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:48.664160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efbb6d-eea4-473a-84ed-46b986f89f14 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:48.704171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:48.704865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:48.705148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:48.705423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:48.781111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:48.782619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:48.808310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:48.889468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 556/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:48.895649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:48.990128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:49.000808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a72d2-fc9d-4109-a447-fb7772a5b0d1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:49.035785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:49.036676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:49.036954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:49.037233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] PATH_INFO: `/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:49.038459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ea902751cc4f04b6f2ecbeeedb3abf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:49.039916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb49d7-b69d-401a-8ca6-492b890840e5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:49.041657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 573/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:48 2022] GET /identity/v3/projects/d6ea902751cc4f04b6f2ecbeeedb3abf => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:49.048661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:49.151534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445be0-86bb-4e00-b3b7-e4d440bc7c6d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:49.175383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:49.189492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:49.190160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:49.190427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:49.190711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:49.191539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] RBAC: Adding query filter params (role.id=6b045d9da54b4f4c8474e262e7326a52, scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf, user.id=3bdf8ba38a9b4714936d1d01671a43b3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:49.191844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:49.193671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059fd28-ede2-4073-be16-889b2c87ddee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:49.199637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 557/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:49 2022] GET /identity/v3/role_assignments?user.id=3bdf8ba38a9b4714936d1d01671a43b3&scope.project.id=d6ea902751cc4f04b6f2ecbeeedb3abf&role.id=6b045d9da54b4f4c8474e262e7326a52 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:51.240304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c1e7-2aad-4e63-8dad-e6f2bede0e1f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:51.240887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c1e7-2aad-4e63-8dad-e6f2bede0e1f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:51.240887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216c1e7-2aad-4e63-8dad-e6f2bede0e1f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:51.242197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 574/1131] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:51.247673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f098965-945e-43df-8748-0287937ada10 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:51.248153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f098965-945e-43df-8748-0287937ada10 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:51.248411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f098965-945e-43df-8748-0287937ada10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:51.266625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2f098965-945e-43df-8748-0287937ada10 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:51.267818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2f098965-945e-43df-8748-0287937ada10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:51.339211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 558/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:51.363297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216c1e7-2aad-4e63-8dad-e6f2bede0e1f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:51.469886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216c1e7-2aad-4e63-8dad-e6f2bede0e1f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:51.509156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:51.509961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:51.510241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:51.510510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:51.511831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:51.513803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a0a70-82d6-4ce1-af89-89aec87a1f9c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:51.515456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 575/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:51.521703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f098965-945e-43df-8748-0287937ada10 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:51.626877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f098965-945e-43df-8748-0287937ada10 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:51.645973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:51.667476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:51.668155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:51.668581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:51.668861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:51.669796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:51.671279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf5c09-ac45-4b1b-9659-3f0fa95f77ea None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:51.701956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 559/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:14:51 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:53.791238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f5b10-368d-43d4-bf74-6424faf888e2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:53.791835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f5b10-368d-43d4-bf74-6424faf888e2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:53.791835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f5b10-368d-43d4-bf74-6424faf888e2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:53.793233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 576/1135] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:53.798689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:53.799151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:53.799252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:53.816596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:53.818161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:53.889501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 560/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:53.914302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f5b10-368d-43d4-bf74-6424faf888e2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:53.946215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:54.027101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f5b10-368d-43d4-bf74-6424faf888e2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:54.062412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:54.063140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:54.063418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:54.063697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:54.064952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:54.066404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cf013-ea2c-4871-89fc-c42418211280 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:54.068426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 577/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:54.074470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:54.178168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:54.182501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624a99-7236-4409-b360-d73726d8aca9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:54.219636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:54.220364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:54.220686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:54.220988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:54.221703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:54.223151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b25ed-f4e3-4cda-8252-c45f21e4b048 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:54.244779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 561/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:14:54 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:14:56.342702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917a5d-3a63-4573-af57-488c77612515 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:56.343273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917a5d-3a63-4573-af57-488c77612515 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:56.343273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917a5d-3a63-4573-af57-488c77612515 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:56.344663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 578/1139] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:56.350036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:56.350470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:56.350568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:56.367841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:56.369094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:56.443662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 562/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:56.468783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6917a5d-3a63-4573-af57-488c77612515 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:56.570513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:56.578985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6917a5d-3a63-4573-af57-488c77612515 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:56.613963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:56.614693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:56.614977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:56.615251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:56.625612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:56.633173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:56.634290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:56.635244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 579/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:56.641328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:56.732584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:56.753071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08c122-a255-456b-a986-5b6d94e2bfc8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:56.791824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:56.792534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:56.792824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:56.793125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:56.793869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:56.794180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:56.795638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:56.804442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 563/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:56.810657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:56.915773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f17c97-75c6-4ee1-9230-370b2e8b5378 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:56.951381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:56.952258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:56.952576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:56.952855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] PATH_INFO: `/v3/users/265d1a966929493bb5f1ef7f4dd641f6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:56.966994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=265d1a966929493bb5f1ef7f4dd641f6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:56.968499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:56.970207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 580/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:14:56 2022] GET /identity/v3/users/265d1a966929493bb5f1ef7f4dd641f6 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:56.976264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:57.087161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e41ad-daa6-4d5e-a5e9-9aec6b6ecf18 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:57.124356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:57.125025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:57.125308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:57.125580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:57.141978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:57.196567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:57.197932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:57.299023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 564/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:14:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:14:57.305320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:57.321492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:57.409195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a5ba9-93e7-43f7-99ee-2b1bbc6bb6e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:57.445592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:57.446470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:57.446817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:57.447026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] PATH_INFO: `/v3/projects/88512892e55f427a8a91b42de0a3720b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:57.448245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] RBAC: Authorizing `identity:get_project(project_id=88512892e55f427a8a91b42de0a3720b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:57.449828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294a4c0-6f50-4a14-a682-79e63b11e481 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:57.451601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 581/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:14:57 2022] GET /identity/v3/projects/88512892e55f427a8a91b42de0a3720b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:57.458066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:57.507861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:57.570829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360cb13-b3d1-4ce3-8c63-378ddf4bfebc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:57.606874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:57.607687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:57.607969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:57.608245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:57.609108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=88512892e55f427a8a91b42de0a3720b, user.id=265d1a966929493bb5f1ef7f4dd641f6) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:14:57.609422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:14:57.610848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf5f8d-9d66-4798-9c3c-98619899a48e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:14:57.616347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 565/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:14:57 2022] GET /identity/v3/role_assignments?user.id=265d1a966929493bb5f1ef7f4dd641f6&scope.project.id=88512892e55f427a8a91b42de0a3720b&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:14:59.626513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618112c4-d1ba-42ec-b926-a376ab6f77d6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:59.627127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618112c4-d1ba-42ec-b926-a376ab6f77d6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:59.627127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618112c4-d1ba-42ec-b926-a376ab6f77d6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:59.628491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 582/1147] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:14:59.634077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:59.634499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:59.634604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:59.652726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:14:59.654111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:14:59.725354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 566/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:14:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:14:59.748768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618112c4-d1ba-42ec-b926-a376ab6f77d6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:59.843945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:14:59.867058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618112c4-d1ba-42ec-b926-a376ab6f77d6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:14:59.902811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:14:59.903520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:14:59.904010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:14:59.904130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:14:59.914466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:14:59.922182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:14:59.923267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:14:59.924269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 583/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:14:59.929875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:14:59.972673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:00.031290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc206e-17e0-44e1-ae92-0d1d040e27ad None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:00.066899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:00.067914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:00.068204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:00.068504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:00.069264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:00.069573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:00.070964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:00.079739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 567/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:00.085892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:00.191535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ea571-04d0-4548-abee-cc2d485e8376 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:00.228538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:00.229418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:00.229697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:00.229974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] PATH_INFO: `/v3/users/265d1a966929493bb5f1ef7f4dd641f6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:00.231222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] RBAC: Authorizing `identity:get_user(user_id=265d1a966929493bb5f1ef7f4dd641f6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:00.233283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:00.234950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 584/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:15:00 2022] GET /identity/v3/users/265d1a966929493bb5f1ef7f4dd641f6 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:00.241030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:00.348275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07393095-5699-4783-80ff-d1fead99aa2e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:00.349895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:00.385083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:00.385959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:00.386244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:00.386518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] PATH_INFO: `/v3/projects/88512892e55f427a8a91b42de0a3720b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:00.387771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] RBAC: Authorizing `identity:get_project(project_id=88512892e55f427a8a91b42de0a3720b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:00.389390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc008b0c-86c8-4261-99c4-2eaa74c80941 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:00.391130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 568/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:15:00 2022] GET /identity/v3/projects/88512892e55f427a8a91b42de0a3720b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:00.397384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:00.409645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:00.503010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f3161-e860-40ee-bd78-a46256b793a6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:00.540002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:00.540931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:00.541207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:00.541472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] PATH_INFO: `/v3/projects/88512892e55f427a8a91b42de0a3720b/users/265d1a966929493bb5f1ef7f4dd641f6/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:00.543553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] RBAC: Authorizing `identity:create_grant(project_id=88512892e55f427a8a91b42de0a3720b, user_id=265d1a966929493bb5f1ef7f4dd641f6, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:00.545429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f96de-dee8-439d-a390-c49a5a7b1154 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:00.560759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 585/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:15:00 2022] PUT /identity/v3/projects/88512892e55f427a8a91b42de0a3720b/users/265d1a966929493bb5f1ef7f4dd641f6/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:15:02.565903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec483f8-42f0-4954-ae91-8343edd40aeb None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:02.566512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec483f8-42f0-4954-ae91-8343edd40aeb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:02.566512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec483f8-42f0-4954-ae91-8343edd40aeb None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:02.568222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 569/1154] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:02.574324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:02.574882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:02.574882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:02.594198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:02.595452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:02.666442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:02.673287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 586/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:02.698614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec483f8-42f0-4954-ae91-8343edd40aeb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:02.809875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec483f8-42f0-4954-ae91-8343edd40aeb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:02.846227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:02.846939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:02.847223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:02.847496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:02.858655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:15:02.866208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:15:02.867448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:15:02.868303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 570/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:15:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:15:02.874185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:02.978323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98873533-e24c-47fc-a3c2-6ae349563a34 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:03.015098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:03.015781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:03.016064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:03.016354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:03.017111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:03.017416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:03.018811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:03.027970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 587/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:03.034350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:03.068500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:03.144610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71bf1fc-bec0-4e32-8116-ebc0cdcdace1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:03.180471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:03.181348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:03.181631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:03.181907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] PATH_INFO: `/v3/users/265d1a966929493bb5f1ef7f4dd641f6` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:03.183109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] RBAC: Authorizing `identity:get_user(user_id=265d1a966929493bb5f1ef7f4dd641f6)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:03.184587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:03.186273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 571/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:15:03 2022] GET /identity/v3/users/265d1a966929493bb5f1ef7f4dd641f6 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:03.192311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:03.255776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:03.301189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19db21-ca1b-4ff8-810a-6c2f1cc9cf83 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:03.337652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:03.338319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:03.338603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:03.338886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:03.411361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:03.412791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:03.472670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:03.518659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 588/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:15:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:15:03.525162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:03.627971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:03.632440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a269f-a43f-4a4d-af71-fe1f1b80855e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:03.668184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:03.669002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:03.669357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:03.669640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] PATH_INFO: `/v3/projects/88512892e55f427a8a91b42de0a3720b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:03.670873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] RBAC: Authorizing `identity:get_project(project_id=88512892e55f427a8a91b42de0a3720b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:03.672366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101baf6f-0cfd-4666-b46e-65425a21eeda None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:03.674085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 572/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:15:03 2022] GET /identity/v3/projects/88512892e55f427a8a91b42de0a3720b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:03.680588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:03.783915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34959026-cfd6-4a89-b8e9-3d2aced58903 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:03.820348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:03.821037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:03.821318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:03.821590 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:03.822429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=88512892e55f427a8a91b42de0a3720b, user.id=265d1a966929493bb5f1ef7f4dd641f6) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:03.822743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:03.824208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408642a-eb0d-44d4-aa28-907a4340f839 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:03.830891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 589/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:15:03 2022] GET /identity/v3/role_assignments?user.id=265d1a966929493bb5f1ef7f4dd641f6&scope.project.id=88512892e55f427a8a91b42de0a3720b&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:05.857037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c696f-d976-4b99-abc5-2142cecf0676 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:05.857603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c696f-d976-4b99-abc5-2142cecf0676 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:05.857603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c696f-d976-4b99-abc5-2142cecf0676 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:05.859056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 573/1162] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:05.865093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:05.865522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:05.865641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:05.884702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:05.885930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:05.906137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:05.957832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 590/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:05.980538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c696f-d976-4b99-abc5-2142cecf0676 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:06.087122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c696f-d976-4b99-abc5-2142cecf0676 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:06.122961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:06.123526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:06.123812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:06.124095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:06.159878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:06.195867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:06.197428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:06.299655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 574/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:15:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:15:06.306061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:06.408094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6337fbe-c57c-4b7f-904a-6fe0ebc851a2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:06.443849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:06.444708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:06.444990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:06.445264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:06.452724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:06.454207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-94511d4c-7abd-43eb-b7ef-ab9bfcd0ae2b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:15:06.469324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:15:06.470520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:15:06.471665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 591/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:15:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:15:06.476631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:06.542177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:06.579929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5238022-ca97-4017-a01e-36aeb55018e3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:06.614443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:06.615113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:06.615397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:06.615673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:06.616424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:06.616734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:06.618115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf18f4-22df-425b-8984-bf4fb87648be None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:06.625532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 575/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jun 27 15:15:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:08.654115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9918a5-dd31-4ff0-909c-0cf7f5ac4b72 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:08.655032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9918a5-dd31-4ff0-909c-0cf7f5ac4b72 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:08.655032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9918a5-dd31-4ff0-909c-0cf7f5ac4b72 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:08.657022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 592/1167] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:08.663925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887f21-0302-40b8-9e29-24f2387d7122 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:08.664743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887f21-0302-40b8-9e29-24f2387d7122 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:08.664916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887f21-0302-40b8-9e29-24f2387d7122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:08.689003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-16887f21-0302-40b8-9e29-24f2387d7122 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:08.690240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-16887f21-0302-40b8-9e29-24f2387d7122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:08.767675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 576/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:08.791063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9918a5-dd31-4ff0-909c-0cf7f5ac4b72 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:08.809119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:08.911139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9918a5-dd31-4ff0-909c-0cf7f5ac4b72 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:08.951575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:08.952243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:08.952571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:08.952849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:08.953596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:08.954994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacd280-e1d1-4a46-b388-808b3cbb8637 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:08.973739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 593/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Jun 27 15:15:08 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:15:11.038801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184ec4-2e11-4912-96b3-52a1af1718e1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:11.039360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184ec4-2e11-4912-96b3-52a1af1718e1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:11.039360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184ec4-2e11-4912-96b3-52a1af1718e1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:11.040859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 577/1170] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:11.046516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:11.046903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:11.047076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:11.064829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:11.066040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:11.141647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 594/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:11.165810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184ec4-2e11-4912-96b3-52a1af1718e1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:11.231630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:11.276556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184ec4-2e11-4912-96b3-52a1af1718e1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:11.312836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:11.313747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:11.314021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:11.314292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] PATH_INFO: `/v3/domains/a7b21cc99d204fcb9e5cc6817ebfc255` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:11.325777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a7b21cc99d204fcb9e5cc6817ebfc255)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:11.327234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00ba79-1175-4053-a579-d7631c7b9a13 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:11.328940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 578/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jun 27 15:15:11 2022] GET /identity/v3/domains/a7b21cc99d204fcb9e5cc6817ebfc255 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:11.335060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:11.338413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:11.438317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ddaf3-9b49-4da4-bb64-ff28361d0527 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:11.474524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:11.475186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:11.475466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:11.475740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:11.476480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:11.477927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c1408-ae97-4a75-931e-5e5e7315c1de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:11.505902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 595/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:15:11 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:15:13.590580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e4d7-c865-46e1-adf1-3ed978f8e918 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:13.591144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e4d7-c865-46e1-adf1-3ed978f8e918 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:13.591144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5e4d7-c865-46e1-adf1-3ed978f8e918 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:13.593083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 579/1174] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:13.598515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:13.598978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:13.599077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:13.617403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:13.618622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:13.672549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:13.691751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 596/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:13.721712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5e4d7-c865-46e1-adf1-3ed978f8e918 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:13.833070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5e4d7-c865-46e1-adf1-3ed978f8e918 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:13.854952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:13.870923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:13.871823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:13.872106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:13.872818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] PATH_INFO: `/v3/domains/a7b21cc99d204fcb9e5cc6817ebfc255` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:13.874068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a7b21cc99d204fcb9e5cc6817ebfc255)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:13.875522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f681a-e862-433f-a960-6d3276cdf569 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:13.877235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 580/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jun 27 15:15:13 2022] GET /identity/v3/domains/a7b21cc99d204fcb9e5cc6817ebfc255 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:13.883231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:13.986114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca6bfd-b662-4eb8-9279-700fffd3fca7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:14.030580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:14.031267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:14.031555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:14.031817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:14.032572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:14.034042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a84ea2-11b5-4b15-ba85-0fbaf770bb91 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:14.058497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 597/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:15:13 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:15:16.178711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdebf-fd6f-4a0b-a3fb-abdbf893ad75 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.179261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdebf-fd6f-4a0b-a3fb-abdbf893ad75 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.179261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fdebf-fd6f-4a0b-a3fb-abdbf893ad75 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:16.180604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 581/1178] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:16.186391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.186839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.186964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:16.205711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:16.206910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:16.244777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:16.278734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 598/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:16.303264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fdebf-fd6f-4a0b-a3fb-abdbf893ad75 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:16.416911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fdebf-fd6f-4a0b-a3fb-abdbf893ad75 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:16.453535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:16.454254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.454533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.454810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:16.467446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:15:16.474869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:15:16.475889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:15:16.476795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 582/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:15:16.482941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:16.587423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bc633-6b17-4485-a1bf-c719ef57e4e5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:16.625169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:16.625831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.626109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.626378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:16.627129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:16.627432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:16.628855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:16.637698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 599/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:16.644020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:16.702772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:16.756337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b639f7-3a04-496a-b564-66c3a94898a6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:16.795273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:16.796146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.796458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.796733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] PATH_INFO: `/v3/users/d38b2ec180cc45f8bc8035b033267cca` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:16.811102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] RBAC: Authorizing `identity:get_user(user_id=d38b2ec180cc45f8bc8035b033267cca)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:16.812601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:16.814280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 583/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:15:16 2022] GET /identity/v3/users/d38b2ec180cc45f8bc8035b033267cca => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:16.820774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:16.860288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:16.929504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46f52-fa74-4321-af11-567fa6d617cd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:16.964340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:16.964998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:16.965278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:16.965567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:17.038053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:17.039421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:17.073480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:17.142328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 600/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:15:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:15:17.148968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:17.257772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03578-5dcc-49f7-85bb-725f0433a955 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:17.265455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:17.295433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:17.296224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:17.296545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:17.296827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] PATH_INFO: `/v3/projects/7d11649c3f7e4df2850100d9b122be4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:17.298159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] RBAC: Authorizing `identity:get_project(project_id=7d11649c3f7e4df2850100d9b122be4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:17.299619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d61912-8d90-4b7b-ab15-e5a9ac2d9b58 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:17.301387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 584/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:15:17 2022] GET /identity/v3/projects/7d11649c3f7e4df2850100d9b122be4d => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:17.307994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:17.416117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0abefb-8767-43ac-a634-ea196b76a07a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:17.452686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:17.453339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:17.453625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:17.453897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:17.454730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=7d11649c3f7e4df2850100d9b122be4d, user.id=d38b2ec180cc45f8bc8035b033267cca) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:17.455047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:17.456528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7209a-ee05-4b95-8f55-7661555ca626 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:17.458373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:17.463515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 601/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:15:17 2022] GET /identity/v3/role_assignments?user.id=d38b2ec180cc45f8bc8035b033267cca&scope.project.id=7d11649c3f7e4df2850100d9b122be4d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:19.463979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe5bc-701b-4e0c-bb24-cb7ab8566b00 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:19.464603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe5bc-701b-4e0c-bb24-cb7ab8566b00 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:19.464603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfe5bc-701b-4e0c-bb24-cb7ab8566b00 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:19.465930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 585/1186] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:19.472044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:19.472610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:19.472610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:19.490390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:19.491597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:19.568196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 602/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:15:19.592748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfe5bc-701b-4e0c-bb24-cb7ab8566b00 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:19.706546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfe5bc-701b-4e0c-bb24-cb7ab8566b00 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:19.743651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:19.744364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:19.744687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:19.744955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:19.755847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:15:19.763424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:15:19.764599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:15:19.765525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 586/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:15:19.771935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:19.881840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1c063-0e36-4120-9de2-2484550ddb47 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:19.910494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:19.923380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:19.924381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:19.924763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:19.925109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:19.926252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:19.926692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:19.928816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:19.939711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 603/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:19.946305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:19.984173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:20.055641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2931f6-2e51-44c4-ab48-641b9037a351 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:20.091567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:20.092451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:20.092742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:20.093017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] PATH_INFO: `/v3/users/d38b2ec180cc45f8bc8035b033267cca` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:20.094322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] RBAC: Authorizing `identity:get_user(user_id=d38b2ec180cc45f8bc8035b033267cca)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:20.095772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:20.097455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 587/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:15:19 2022] GET /identity/v3/users/d38b2ec180cc45f8bc8035b033267cca => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:15:20.104305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:20.220355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c92034-2cc2-49ac-9ac0-29608f74bef1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:20.258174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:20.259034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:20.259313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:20.259581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] PATH_INFO: `/v3/projects/7d11649c3f7e4df2850100d9b122be4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:20.260906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] RBAC: Authorizing `identity:get_project(project_id=7d11649c3f7e4df2850100d9b122be4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:20.262353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0be2a-b8c8-454a-beed-d7b57c03e510 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:20.264630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 604/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:15:20 2022] GET /identity/v3/projects/7d11649c3f7e4df2850100d9b122be4d => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:20.270987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:20.392252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9b8-00a8-4b6d-93b6-0532114883c4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:20.393797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:20.429108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:20.429998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:20.430278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:20.430552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] PATH_INFO: `/v3/projects/7d11649c3f7e4df2850100d9b122be4d/users/d38b2ec180cc45f8bc8035b033267cca/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:20.432608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d11649c3f7e4df2850100d9b122be4d, user_id=d38b2ec180cc45f8bc8035b033267cca, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:20.434055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a539ef-66b4-443e-a4dc-aa4099104b83 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:20.450043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 588/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:15:20 2022] PUT /identity/v3/projects/7d11649c3f7e4df2850100d9b122be4d/users/d38b2ec180cc45f8bc8035b033267cca/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:15:22.459917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3aedd-163c-4a23-9217-ccb3aadd28f7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:22.460508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3aedd-163c-4a23-9217-ccb3aadd28f7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:22.460508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3aedd-163c-4a23-9217-ccb3aadd28f7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:22.461896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 605/1193] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:15:22.468265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:22.468812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:22.468909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:22.487242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:15:22.488487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:15:22.569001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 589/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:15:22.602071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3aedd-163c-4a23-9217-ccb3aadd28f7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:22.666822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:22.712520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3aedd-163c-4a23-9217-ccb3aadd28f7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:22.751501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:22.752474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:22.752756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:22.753066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:22.764766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:15:22.772484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:15:22.780722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:15:22.782360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 606/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:15:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:15:22.783652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:22.891927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:22.902712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa783c3-2998-48d6-8cd7-862ceef00342 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:22.939585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:22.940427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:22.940704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:22.941027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:22.941832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:22.942179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:22.943978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:22.953497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 590/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:15:22.959689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:23.062284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c23fa-6a33-4a5e-8e90-6f80b726adfb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:23.080584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:23.098927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:23.099811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:23.100090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:23.100385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] PATH_INFO: `/v3/users/d38b2ec180cc45f8bc8035b033267cca` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:23.101616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] RBAC: Authorizing `identity:get_user(user_id=d38b2ec180cc45f8bc8035b033267cca)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:23.103101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:23.104831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 607/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jun 27 15:15:22 2022] GET /identity/v3/users/d38b2ec180cc45f8bc8035b033267cca => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:15:23.110891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:23.221531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d025ad-d329-4b94-b739-f0d06bca3eb7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:23.257508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:23.258206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:23.258488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:23.258768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:23.288782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:23.356165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:23.358974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:23.476756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 591/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:15:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:15:23.483075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:23.595794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379db96-f250-409c-8ebe-f538fe6f0f79 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:23.636441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:23.637267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:23.637548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:23.637822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] PATH_INFO: `/v3/projects/7d11649c3f7e4df2850100d9b122be4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:23.639314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d11649c3f7e4df2850100d9b122be4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:23.640836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa90dd-2686-4911-a7c0-56cc0fed59b9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:23.642689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 608/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jun 27 15:15:23 2022] GET /identity/v3/projects/7d11649c3f7e4df2850100d9b122be4d => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:23.649557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:15:23.735414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:23.789564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf0afe-b31a-438f-a871-cbb6cb378003 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:15:23.829751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:15:23.830465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:15:23.830783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:15:23.831098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:15:23.832002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=7d11649c3f7e4df2850100d9b122be4d, user.id=d38b2ec180cc45f8bc8035b033267cca) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:15:23.832358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:15:23.833918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e933d44-66b1-4138-9585-84e93acc5d5a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:15:23.840850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 592/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:15:23 2022] GET /identity/v3/role_assignments?user.id=d38b2ec180cc45f8bc8035b033267cca&scope.project.id=7d11649c3f7e4df2850100d9b122be4d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:15:33.631202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=73 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:15:33.838404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=41 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:22.832658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaba6b-7265-4871-8cce-bc4cd637e19d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:22.833630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaba6b-7265-4871-8cce-bc4cd637e19d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:22.833968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaba6b-7265-4871-8cce-bc4cd637e19d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:22.835536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 609/1201] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:22.846380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:22.846380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:22.846380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:22.875453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:22.875453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:23.007528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 593/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:16:23.036877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaba6b-7265-4871-8cce-bc4cd637e19d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:23.232855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaba6b-7265-4871-8cce-bc4cd637e19d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:23.290101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:23.291006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:23.291353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:23.291670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:23.305855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:23.309966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:23.333312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:23.334385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:23.335409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:23.335409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:23.335409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:23.335409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:23.335409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 610/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:23.343857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:23.490550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6917b-1562-4ac0-a87a-51dfb8cea382 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:23.541876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:23.549210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:23.549506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:23.549751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:23.550029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:23.550795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:23.551115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:23.553766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287d92-bb14-40e9-98c7-6454bede0b31 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:23.568845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 594/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:23.574470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:23.644180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:23.703631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fd339-94bc-49c2-9235-7b419e61686a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:23.748357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:23.749427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:23.749947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:23.750429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:23.751357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:23.753066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a4cf-bc11-45bd-9e40-754bb7de866d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:23.780363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 611/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jun 27 15:16:23 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:16:26.075297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656cae-a9fd-411b-a215-efa06e778abc None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:26.075967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656cae-a9fd-411b-a215-efa06e778abc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:26.076053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656cae-a9fd-411b-a215-efa06e778abc None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:26.077528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 595/1206] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:26.084774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:26.084774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:26.084774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:26.103484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:26.104761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:26.194546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 612/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:26.220086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60656cae-a9fd-411b-a215-efa06e778abc None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:26.333270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60656cae-a9fd-411b-a215-efa06e778abc None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:26.370027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:26.370824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:26.371103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:26.371372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:26.390681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:26.399882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:26.401496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:26.402409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 596/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:26.408785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:26.463076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:26.544093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b7912-f7d4-406a-8cdb-c98ba0b5413d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:26.593096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:26.593903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:26.594276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:26.594465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:26.595246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:26.595553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:26.597012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:26.607303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 613/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:26.613650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:26.682580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:26.749286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4d101-8a15-44a9-b49d-e7867dd0c5cc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:26.790832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:26.791644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:26.791926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:26.792201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:26.804160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:26.805923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:26.826811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:26.828268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:26.829382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:26.829382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:26.829382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:26.829382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:26.829382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 597/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:26.833898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:26.983963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:26.987047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051d9b3-23a5-478e-8bd3-8a3ae5a54189 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:27.024571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:27.025255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:27.025534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:27.025808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:27.026532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:27.026835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:27.028283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:27.045518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 614/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:27.046869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:27.160946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd848f5-b4b0-4a2d-b57e-dc828b9b2900 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:27.162877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:27.202016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:27.202907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:27.203312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:27.203611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:27.213080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:27.214758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:27.232020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:27.234784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:27.236867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 598/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:27.239527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:27.366508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0819a3-c33c-44a7-a421-37b136f355f5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:27.406560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:27.407304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:27.407605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:27.407894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:27.408901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:27.409211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:27.410851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:27.430445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 615/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:27.436777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:27.559385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e3299-38ef-4d15-bb5e-73d6e5be66d1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:27.599637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:27.600429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:27.601021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:27.601021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:27.636792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:27.689859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:27.692882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:27.824376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 599/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:16:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:16:27.829479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:27.854515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:27.994978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edde1e0-dcc9-47d8-8294-3e06062245a8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:28.046213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:28.047238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:28.047612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:28.047890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:28.061339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:28.062859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:28.090191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:28.091892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:28.098234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:28.098234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:28.098234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:28.098234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:28.098234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 616/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:16:28.098975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:28.180102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:28.235549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924811c-e763-4709-aa9a-7bd816d6d7f4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:28.276695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:28.277391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:28.277677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:28.277999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:28.278846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:28.278846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:28.280564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:28.294031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 600/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:28.299773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:28.399820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:28.445156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9103ec5-51a5-4f99-9974-4c58aa6e368a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:28.500220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:28.501445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:28.501957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:28.502199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:28.513910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:28.516517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:28.534255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:28.535436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:28.536587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:28.537127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 617/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:28.543407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:28.668450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78147b80-c67b-4993-9af0-235228aec0e6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:28.702309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:28.715261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:28.716023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:28.716620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:28.716620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:28.718939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:28.719254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:28.720930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719ffef-a84d-4034-8b97-811f5b132910 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:28.740772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 601/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:28.746975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:28.895373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d770-288f-4ade-a281-34d5d219e69c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:28.940006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:28.951353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:28.952024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:28.952309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:28.952619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:28.953452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=26b59442ce5949a59dfe58d1b4b98e6e) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:28.953771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:28.955345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d99caa-9be9-4830-9764-9a8301ce6d4e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:28.964970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 618/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:16:28 2022] GET /identity/v3/role_assignments?user.id=26b59442ce5949a59dfe58d1b4b98e6e&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:31.041658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ebfc-59d4-45a8-a292-ba3d991cf5ff None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.042233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ebfc-59d4-45a8-a292-ba3d991cf5ff None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.042233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4ebfc-59d4-45a8-a292-ba3d991cf5ff None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.043693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 602/1220] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:31.049786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.050304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.050304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.069030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:31.070246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:31.154618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 619/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:31.177994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4ebfc-59d4-45a8-a292-ba3d991cf5ff None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:31.319139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4ebfc-59d4-45a8-a292-ba3d991cf5ff None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:31.357985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:31.358905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.359074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.359347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.370150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:31.378387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:31.379685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:31.380930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 603/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:16:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:31.386121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:31.497044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c615d-4dfd-4a7a-942a-2d2ddee74b00 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:31.523653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:31.534928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:31.535608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.535888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.536161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.536945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:31.537250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:31.538636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:31.549142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 620/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:16:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:31.555093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:31.612937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:31.694832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c48f-a2c6-43b1-ae25-f74d24421f13 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:31.730314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:31.731120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.731568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.732426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.744799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:31.746298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:31.766873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:31.768084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:31.774006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:31.774006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:31.774006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:31.774006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:31.774006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 604/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:31.774344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:31.912612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34acc9-5554-4c00-8fdb-3bd65190f790 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:31.952609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:31.953292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:31.953578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:31.953858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:31.954602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:31.954916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:31.956341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:31.969233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 621/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:31.976394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:32.080776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:32.088115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04deeb10-d909-4cb6-af7f-871617e03ff3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:32.125628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:32.126470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:32.126746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:32.127021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:32.135588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:32.137123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:32.152636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:32.153841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:32.154893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 605/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:16:32.161907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:32.218058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:32.275408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6e3bb-b809-4a9e-a167-fbef6ba5070f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:32.324348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:32.324348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:32.324348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:32.324348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:32.324851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:32.324851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:32.324851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:32.345772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 622/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:32.352434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:32.469701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f381fc-871f-4d3c-8c63-99928b8d27cd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:32.507872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:32.508799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:32.509080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:32.509351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:32.523951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:32.526103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:32.546554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:32.547945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:32.549039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:32.549039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:32.549039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:32.549039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:32.549039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 606/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:32.554120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:32.671771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda0a4-290e-470a-9c54-5611d8877d90 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:32.678159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:32.721063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:32.721792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:32.722073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:32.722377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:32.723253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:32.723586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:32.725142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:32.739187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 623/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:32.745198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:32.756038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:32.880207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480910-30c2-45bb-a70d-4d25a56c5362 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:32.924276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:32.925048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:32.925360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:32.925609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:32.933894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:32.935880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:32.952218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:32.953660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:32.954970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:32.958519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 607/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:32.961008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:33.087547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075fc73-c313-47c3-a0c7-24c7d901beec None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:33.127655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:33.128347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:33.128635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:33.128908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:33.129695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:33.130002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:33.131502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bd534-26d3-4b24-9d60-31007b9cc337 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:33.146935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 624/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:33.153230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:33.226504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:33.276803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176f3bc-2c29-4aa5-8018-b23aea779735 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:33.329588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:33.330523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:33.330800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:33.331075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/26b59442ce5949a59dfe58d1b4b98e6e/roles/9ed69811fed94bba95ca24e74e17a61a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:33.352612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=26b59442ce5949a59dfe58d1b4b98e6e, role_id=9ed69811fed94bba95ca24e74e17a61a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:33.354548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24010708-1a5d-4b8b-925c-bb68e40fe88e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:33.374424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 608/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:16:33 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/26b59442ce5949a59dfe58d1b4b98e6e/roles/9ed69811fed94bba95ca24e74e17a61a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:16:35.704055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ccb2-6392-466f-8b6a-b54ddec07877 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:35.704696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ccb2-6392-466f-8b6a-b54ddec07877 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:35.704696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62ccb2-6392-466f-8b6a-b54ddec07877 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:35.706007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 625/1233] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:35.711744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:35.712356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:35.712356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:35.733208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:35.734426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:35.821931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 609/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:35.847595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62ccb2-6392-466f-8b6a-b54ddec07877 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:35.883398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:35.990822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62ccb2-6392-466f-8b6a-b54ddec07877 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:36.047659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:36.048604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:36.048897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:36.049180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:36.067697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:36.075388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:36.076482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:36.077349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 626/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jun 27 15:16:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:36.082929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:36.131901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:36.190312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03184a-0466-4b9e-8263-a566b1d93827 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:36.226488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:36.228109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:36.228456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:36.228704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:36.229534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:36.229838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:36.231258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:36.240276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 610/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jun 27 15:16:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:36.246351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:36.342832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:36.363338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d01602-9e20-4d03-a4f2-eebd4ac22797 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:36.404434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:36.405196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:36.405471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:36.405747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:36.417795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:36.419612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:36.440103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:36.442010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:36.444193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:36.444193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:36.444193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:36.444193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:36.444193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 627/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:36.448878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:36.578271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582979-0c6f-434c-96fd-ee31525c87e7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:36.588653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:36.613851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:36.614527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:36.614814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:36.615092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:36.615819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:36.616128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:36.617553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:36.630382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 611/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:36.636712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:36.756835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff05b89-1963-4141-92c7-e5121a89db29 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:36.823180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:36.825285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:36.825834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:36.826528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:36.833116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:36.847764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:36.852048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:36.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:36.898262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:36.899808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:36.900584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 628/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:16:36.907890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:37.054972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718abb7-4c3b-46a6-83b6-192ae643f383 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:37.093686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:37.094291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:37.094576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:37.094887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:37.095719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:37.096029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:37.097516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:37.117590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 612/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:37.125421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:37.261832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e58171-1b16-4a84-ab51-a1bdaa61ca38 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:37.300424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:37.301607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:37.301607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:37.301607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:37.388182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:37.396307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:37.397810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:37.541407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 629/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:16:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:16:37.547409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:37.586028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:37.692989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8dabd-f65f-4fc6-9d23-25682493ebdd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:37.740166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:37.741090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:37.741368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:37.741606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:37.756719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:37.758247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:37.785005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:37.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:37.787138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:37.787138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:37.787138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:37.787138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:37.787138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 613/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:37.795787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:37.926373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:37.933964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248b44-90f2-42d0-b79c-d49897e26640 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:37.978864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:37.979704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:37.980091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:37.980506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:37.981632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:37.982084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:37.984109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:38.002817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 630/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:38.008352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:38.148993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:38.179798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b4198-ec19-49b3-bd91-833ffb2c37d6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:38.230403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:38.231249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:38.231741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:38.232074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:38.240394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:38.241906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:38.258547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:38.260163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:38.261779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 614/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:38.266255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:38.393224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056272c-b9c5-416e-a5df-900a643e6f96 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:38.431344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:38.432021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:38.432301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:38.432600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:38.433388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:38.433692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:38.435358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f050ea-8760-489c-8537-91ff6b676e39 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:38.451938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 631/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:38.456882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:38.611064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fb06e-632a-4e19-8687-4de40d668240 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:38.655069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:38.655773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:38.656056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:38.656458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:38.657186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] RBAC: Adding query filter params (role.id=9ed69811fed94bba95ca24e74e17a61a, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=26b59442ce5949a59dfe58d1b4b98e6e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:38.657495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:38.658974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa92e63-accb-48e9-a81c-53923c3f8902 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:38.666225 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 615/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:16:38 2022] GET /identity/v3/role_assignments?user.id=26b59442ce5949a59dfe58d1b4b98e6e&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=9ed69811fed94bba95ca24e74e17a61a => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:41.205277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b937f-816a-49c3-bf41-b23cb7b08e24 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:41.206176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b937f-816a-49c3-bf41-b23cb7b08e24 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:41.206176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b937f-816a-49c3-bf41-b23cb7b08e24 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:41.208379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 632/1247] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:41.215118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105283-8c01-4897-a719-13984537c635 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:41.215939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105283-8c01-4897-a719-13984537c635 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:41.215939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105283-8c01-4897-a719-13984537c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:41.223508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:41.237808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-37105283-8c01-4897-a719-13984537c635 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:41.239025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-37105283-8c01-4897-a719-13984537c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:41.336379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 616/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:41.369038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b937f-816a-49c3-bf41-b23cb7b08e24 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:41.398731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:41.519345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b937f-816a-49c3-bf41-b23cb7b08e24 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:41.573295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:41.574115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:41.574399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:41.574675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:41.590717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:41.600852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:41.601846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:41.603051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 633/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:41.609232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37105283-8c01-4897-a719-13984537c635 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:41.731955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37105283-8c01-4897-a719-13984537c635 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:41.771650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:41.779424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:41.780131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:41.780447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:41.780731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:41.781502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:41.781815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:41.783217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:41.801840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 617/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:41.803262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:41.910790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:41.947756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2a1ec-4c44-4bd7-bb90-6c1da4bda7a5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:41.997731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:41.998767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:41.999053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:41.999331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:42.012201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:42.015039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:42.043393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:42.044587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:42.045644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:42.045644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:42.045644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:42.045644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:42.045644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 634/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:42.054181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:42.206037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a851b-a92b-43d8-8924-f0687e6a9ea3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:42.261101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:42.262069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:42.262419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:42.262883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:42.264259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:42.264927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:42.267864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:42.287856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 618/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:42.293879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:42.430600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3fc8d-83cd-4c82-9f9c-7856cf3649ee None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:42.472165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:42.485810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:42.486764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:42.487075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:42.487348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:42.498064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:42.499885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:42.526308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:42.527884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:42.529315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:42.532426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 635/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:42.538771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:42.609676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:42.698625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a23a25-4568-466f-ac9f-acc8d55b8eaa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:42.758033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:42.759082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:42.759528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:42.759909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:42.761248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:42.762884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:42.764343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:42.793840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 619/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:42.801517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:42.964767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c650fd7-5b92-44ce-a268-d8a47e9732e5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:43.012092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:43.013266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:43.013692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:43.014117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:43.092688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:43.108840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:43.110886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:43.252561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 636/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:16:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:16:43.259113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:43.414241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678eecd-8c98-4855-95d2-ea68b8e5f65d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:43.428546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:43.455951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:43.456830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:43.457096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:43.457371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:43.470876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:43.473038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:43.492688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:43.493894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:43.494851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:43.494851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:43.494851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:43.494851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:43.494851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 620/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:43.503652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:43.622780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:43.639135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944966fc-5483-44a1-8a3c-6bb8995d0202 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:43.682241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:43.683120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:43.683481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:43.683843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:43.684642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:43.684957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:43.686363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:43.700474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 637/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:43.707217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:43.856249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29979607-d9df-4932-abdb-d717f7b5a7b0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:43.907619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:43.908851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:43.909175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:43.909518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:43.917967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:43.919702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:43.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:43.936030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:43.937181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 621/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:16:43.944173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:44.057514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2fe7-c892-46b5-b9e3-2aa280d5ff0c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:44.093880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:44.094655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:44.094893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:44.095323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:44.096433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:44.096856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:44.098487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809973a-45aa-4a44-afec-537f8952f106 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:44.114306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 638/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:44.121031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:44.161097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:44.265887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb347-e514-4af6-930a-7c10ca8261a2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:44.305172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:44.305855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:44.306131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:44.306409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:44.307276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=26b59442ce5949a59dfe58d1b4b98e6e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:44.307641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:44.309403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282316f3-ff9e-4acb-86e7-1575f7bff1bb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:44.329465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 622/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:16:44 2022] GET /identity/v3/role_assignments?user.id=26b59442ce5949a59dfe58d1b4b98e6e&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:46.440638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b75fb-a228-4e9c-9f03-d150910361fe None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:46.441227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b75fb-a228-4e9c-9f03-d150910361fe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:46.441227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b75fb-a228-4e9c-9f03-d150910361fe None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:46.442963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 639/1261] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:46.448494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:46.448950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:46.449041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:46.467372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:46.468625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:46.535592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 623/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:46.558563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b75fb-a228-4e9c-9f03-d150910361fe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:46.570322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:46.663273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b75fb-a228-4e9c-9f03-d150910361fe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:46.701424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:46.702135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:46.702418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:46.702696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:46.714956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:46.722676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:46.723798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:46.725469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 640/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:46.734051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:46.763873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:46.863188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8927f29-0e86-4261-b302-f72a7446c08b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:46.899004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:46.899685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:46.900024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:46.900196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:46.900968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:46.901273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:46.902665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:46.911479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 624/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:16:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:46.919824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:47.000219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:47.046716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd488a-4071-4da3-a9d2-f4cbae408375 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:47.090632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:47.092234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:47.092542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:47.092818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:47.103994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:47.105496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:47.124157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:47.125374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:47.126399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:47.126399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:47.126399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:47.126399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:47.126399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 641/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:47.131444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:47.226739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:47.250337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee571e-f13a-4f64-8367-af89e8a1838b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:47.292229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:47.292994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:47.293276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:47.293551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:47.294690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:47.294690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:47.296057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:47.309189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 625/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:47.316134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:47.437667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577d934-4014-4d87-ab42-7f8b49db3be8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:47.462715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:47.480365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:47.481523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:47.482683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:47.483151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:47.494290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:47.495761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:47.510846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:47.511932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:47.513160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 642/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:47.519227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:47.630869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce6730-b065-41b6-9256-1642cd7f0321 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:47.666915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:47.667625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:47.667912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:47.668197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:47.669262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:47.669673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:47.671127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:47.681018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:47.691812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 626/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:47.697956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:47.819936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf1d45-2905-42c2-a4a0-28e26a7ab58f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:47.855543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:47.856427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:47.856763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:47.857324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:47.868970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:47.870442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:47.889858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:47.891014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:47.892083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:47.892083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:47.892083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:47.892083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:47.892083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 643/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:47.897145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:48.010131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c659fee-1556-454b-bc39-7adf67ac06d8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:48.050472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:48.051162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:48.051443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:48.051699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:48.052453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:48.052761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:48.054162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:48.074770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 627/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:48.084830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:48.124218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:48.203466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7ae8-4f14-4a88-b117-7a2a1988f774 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:48.240647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:48.241441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:48.241723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:48.241997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:48.249580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:48.251048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:48.266202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:48.267205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:48.268244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 644/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:48.273596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:48.342014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:48.385209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9d7a3-a7c4-4405-8cc0-bb00bb7c1d48 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:48.420217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:48.420907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:48.421189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:48.421478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:48.422262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:48.422572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:48.424005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc1924-10ed-4f26-88a4-4f49dc0fc0f3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:48.437035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 628/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:48.443623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:48.537149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:48.546596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c20bae-18ce-42d1-89c0-7a1b63767384 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:48.582363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:48.583247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:48.583522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:48.583797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] PATH_INFO: `/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/26b59442ce5949a59dfe58d1b4b98e6e/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:48.585846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b85fab3471384942986b8c2cd35c2b9d, user_id=26b59442ce5949a59dfe58d1b4b98e6e, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:48.587312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a64fe-07fe-4670-ab96-896e7f8cd2f8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:48.612390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 645/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jun 27 15:16:48 2022] PUT /identity/v3/projects/b85fab3471384942986b8c2cd35c2b9d/users/26b59442ce5949a59dfe58d1b4b98e6e/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:16:50.627328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d75f0-d031-4d5d-9f31-04ac2016fa04 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:50.627963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d75f0-d031-4d5d-9f31-04ac2016fa04 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:50.627963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d75f0-d031-4d5d-9f31-04ac2016fa04 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:50.629281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 629/1274] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:50.634690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:50.635175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:50.635272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:50.653862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:50.655095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:50.730239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 646/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:50.754282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d75f0-d031-4d5d-9f31-04ac2016fa04 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:50.863581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d75f0-d031-4d5d-9f31-04ac2016fa04 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:50.884052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:50.903921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:50.904771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:50.905059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:50.905341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:50.916310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:16:50.924071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:50.925134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:50.926060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 630/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jun 27 15:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:50.931659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.016431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:51.036005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3486d-9d34-4b6d-aadc-1d18ec657495 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.073012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.073690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.073971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.074249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:51.075041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:51.075350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:51.076768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:51.085570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 647/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jun 27 15:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:51.091652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.212058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac14f69-b40f-49ae-b830-dc8c7a34e48c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.252941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.253719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.254064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.254255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:51.266772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:51.268311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:51.286407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:51.287498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:51.293362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:51.293362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:51.293362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:51.293362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:51.293362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 631/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:51.293951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.402190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200e7c-cd48-4981-9645-cadb44c82028 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.440681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.441353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.441635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.441908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:51.442999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:51.443301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:51.444721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:51.457533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 648/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:51.463528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.499450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:51.574868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c567c1-766e-4fa4-8352-05b60d9b9091 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.610730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.611523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.611827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.612089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:51.620284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:51.621789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:16:51.636486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:51.638669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:51.639859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 632/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:51.644097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.648079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:51.749123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60dfa6-6d1e-4661-9725-7438112501d6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.790466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.791138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.791494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.791755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:51.792624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:51.792913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:51.795078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:51.814161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 649/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jun 27 15:16:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:51.820525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:51.917350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:51.927973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01b3d9-1d3e-4878-bd1a-f6d78ffd65c5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:51.963929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:51.964621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:51.964903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:51.965180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.048373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.049927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:52.146763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:52.158009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 633/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jun 27 15:16:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:16:52.164094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:52.234462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:52.273959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade48b-9730-4767-8cc7-9218f0f779b0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:52.309507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:52.310553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:52.310833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:52.311100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.321681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.323133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:16:52.341439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:52.342450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:52.343372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:16:52.343372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:16:52.343372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:16:52.343372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:52.343372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 650/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:52.348616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:52.451284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dbb720-2c0b-499b-a07c-6977608901bb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:52.486448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:52.487117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:52.487393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:52.487680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.488434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:52.488747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.490132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:52.501577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 634/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jun 27 15:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:52.507535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:52.614954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2e990-93b9-4945-a1b2-fc0dd02a4e27 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:52.629484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:52.651681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:52.652521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:52.652803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:52.653075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.660583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.662047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:16:52.675834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:52.676986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jun 27 15:16:52.677929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 651/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jun 27 15:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:52.683083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:52.737024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:52.789245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354b4b4-8b34-4fe9-a401-50c82a9a9725 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:52.825665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:52.826341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:52.826618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:52.826891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.827682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:52.827987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.829450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2ceac-26c8-4252-9390-536c1ee327a9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:52.843377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 635/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jun 27 15:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:52.849719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:52.959456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d2cec-9896-4b56-8b80-e14b9bba9486 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:52.994919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:52.995556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:52.995854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:52.996141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:52.997026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] RBAC: Adding query filter params (role.id=abfc6b6d02e0407bbaef10a05d71ba62, scope.project.id=b85fab3471384942986b8c2cd35c2b9d, user.id=26b59442ce5949a59dfe58d1b4b98e6e) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:52.997347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:52.998798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666907-07e5-44a6-9607-a769d30d9e66 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:53.005117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 652/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jun 27 15:16:52 2022] GET /identity/v3/role_assignments?user.id=26b59442ce5949a59dfe58d1b4b98e6e&scope.project.id=b85fab3471384942986b8c2cd35c2b9d&role.id=abfc6b6d02e0407bbaef10a05d71ba62 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:55.061736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c13-800e-4daa-bf3c-b4bfc70264ec None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:55.062339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c13-800e-4daa-bf3c-b4bfc70264ec None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:55.062339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3755c13-800e-4daa-bf3c-b4bfc70264ec None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:55.063675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 636/1288] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:55.070691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:55.071200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:55.071609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:55.091215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:55.092451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:55.126109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:55.167325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 653/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:55.212772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755c13-800e-4daa-bf3c-b4bfc70264ec None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:55.343822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3755c13-800e-4daa-bf3c-b4bfc70264ec None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:55.355497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:55.383410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:55.384135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:55.384453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:55.384733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] PATH_INFO: `/v3/services/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:55.385398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:55.386798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:16:55.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:16:55.401843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:16:55.402898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 637/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:16:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:16:55.409042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:55.539376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912738ec-b4fa-402e-871d-be55bdae465f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:55.581058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:55.581058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:55.581058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:55.581058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:55.581624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:55.581624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:55.584166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd420c-0d1b-4904-ab26-ed1d6cb7e4bf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:55.587956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 654/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jun 27 15:16:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:16:55.594694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:55.703316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee0c7-45b7-4b43-9540-8a15552d47de None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:55.746426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:55.747410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:55.747410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:55.747694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:55.748436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] RBAC: Adding query filter params (type=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:16:55.748746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:55.750128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd679157-67f0-4943-846b-fd03bc1557dc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:55.755328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 638/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jun 27 15:16:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:16:57.772911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5adb2-0e17-41d4-a071-6b69a69e26e5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:57.773519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5adb2-0e17-41d4-a071-6b69a69e26e5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:57.773519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5adb2-0e17-41d4-a071-6b69a69e26e5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:57.774823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 655/1293] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:16:57.780547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439fb6-043e-44b7-b8ed-457e0c1f5dc5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:57.780982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439fb6-043e-44b7-b8ed-457e0c1f5dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:57.781098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93439fb6-043e-44b7-b8ed-457e0c1f5dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:57.799486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-93439fb6-043e-44b7-b8ed-457e0c1f5dc5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:16:57.800755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-93439fb6-043e-44b7-b8ed-457e0c1f5dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:16:57.838783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:57.882221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 639/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:16:57.906660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5adb2-0e17-41d4-a071-6b69a69e26e5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:16:57.939393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:16:58.014787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5adb2-0e17-41d4-a071-6b69a69e26e5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:16:58.050410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:16:58.050997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:16:58.051305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:16:58.051554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:16:58.052287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:16:58.053718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117b6a-0a15-40ae-8bab-8b2a8eff76d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:16:58.069581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 656/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jun 27 15:16:57 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:17:00.115514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339980e9-ef3f-44fa-aed6-69231a0bca7d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.116111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339980e9-ef3f-44fa-aed6-69231a0bca7d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.116111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339980e9-ef3f-44fa-aed6-69231a0bca7d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.117421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 640/1296] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:17:00.123222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.123686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.123798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.141123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:17:00.142324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:17:00.219728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 657/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:17:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:17:00.245003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339980e9-ef3f-44fa-aed6-69231a0bca7d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:00.357518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339980e9-ef3f-44fa-aed6-69231a0bca7d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:00.393453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:00.394165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.394445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.394715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.395362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:00.396779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:17:00.409516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:17:00.410649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:00.411492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 641/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:17:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:17:00.417164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:00.437301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:00.521698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b2db8-7f08-4666-a4da-498aca998bd0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:00.565284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:00.565962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.566239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.566516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.567202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:17:00.567885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:00.569309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:00.574850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 658/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jun 27 15:17:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:17:00.581224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:00.617761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:00.699424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ac406-d302-4d04-9097-757d7243338a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:00.736720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:00.737390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.737671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.737941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.738738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] RBAC: Adding query filter params (interface=public, service_id=742368e517fa4e21a08f7b65161e19f5, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:17:00.739048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:00.740452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df976-c1f3-4224-94d9-9341fe67cfdd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:00.746174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 642/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jun 27 15:17:00 2022] GET /identity/v3/endpoints?service_id=742368e517fa4e21a08f7b65161e19f5&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:17:00.752371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:00.835826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:00.855669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ea39c-2ed4-4efc-b810-5452fd68df9e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:00.891195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:00.891859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:00.892142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:00.892449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:00.893127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:00.894509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e929a-4f9b-4576-9423-7899f6c038a1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:00.901333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 659/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jun 27 15:17:00 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:17:02.965263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2177-29a9-41c9-b23c-982dc50085cd None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:02.965947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2177-29a9-41c9-b23c-982dc50085cd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:02.966146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2177-29a9-41c9-b23c-982dc50085cd None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:02.967664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 643/1302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:17:02.975648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:02.976746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:02.977027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:02.998346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:17:02.999576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:17:03.088487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 660/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:17:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:17:03.112692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2177-29a9-41c9-b23c-982dc50085cd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:03.225253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:03.233377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2177-29a9-41c9-b23c-982dc50085cd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:03.269728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:03.270529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:03.270816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:03.271097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:03.271755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:03.273173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:17:03.285280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:17:03.286426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Jun 27 15:17:03.287601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 644/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jun 27 15:17:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:17:03.292700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:03.392383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:03.418161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadcd91-07ed-447c-92fe-0b7830a68999 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:03.453640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:03.454303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:03.454583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:03.454856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:03.455554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:17:03.455857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:03.457264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac02ce9-36fc-4702-8ec8-e816727a81dd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:03.464516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 661/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jun 27 15:17:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:17:03.471123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:17:03.581521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eeaf3-13bd-4b8c-98af-8d93a6f8cb56 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:17:03.617702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:17:03.618359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:17:03.618636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:17:03.618909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:17:03.619626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:17:03.621020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f339f-851a-48fe-b0dd-5987bdb7b74a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:17:03.640931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 645/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jun 27 15:17:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:17:13.461405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=30 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:17:13.635921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:12.876009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edaba3-fa84-4d2e-b5a9-dee5c7aea0b9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:12.876518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edaba3-fa84-4d2e-b5a9-dee5c7aea0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:12.876765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edaba3-fa84-4d2e-b5a9-dee5c7aea0b9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:12.882764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 662/1307] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:12.884238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10517c50-f85b-4683-8b1c-09e16b0fe5f8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:12.884889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10517c50-f85b-4683-8b1c-09e16b0fe5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:12.884889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10517c50-f85b-4683-8b1c-09e16b0fe5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:13.066582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-10517c50-f85b-4683-8b1c-09e16b0fe5f8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:13.068204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-10517c50-f85b-4683-8b1c-09e16b0fe5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:13.277784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 646/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:13.318897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:13.319207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:13.319576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:13.321885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 663/1309] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jun 27 15:18:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:13.329581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3360-78c8-4a3b-9883-a7277c902534 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:13.329877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3360-78c8-4a3b-9883-a7277c902534 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:13.330157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3360-78c8-4a3b-9883-a7277c902534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:13.449183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9f7d3360-78c8-4a3b-9883-a7277c902534 None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:13.452065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f7d3360-78c8-4a3b-9883-a7277c902534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:13.483643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:13.552418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 647/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jun 27 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:13.562488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:13.564763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:13.711560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19c344-278b-403c-a92c-9928fe7b780f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:13.831309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:13.832703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:13.833271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:13.833744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:13.835746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:13.912212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:13.981433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:13.983234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bde164-3514-4162-afd9-d57c7d8b1912 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:14.006558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 664/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:16.582054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a77f-3ed4-413b-a563-c0a5de1ea3f4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:16.582659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a77f-3ed4-413b-a563-c0a5de1ea3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:16.582659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a77f-3ed4-413b-a563-c0a5de1ea3f4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:16.584028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 648/1312] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:16.590181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba7e4-5e2b-4055-9c86-0de7b82c782e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:16.591604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba7e4-5e2b-4055-9c86-0de7b82c782e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:16.591604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eba7e4-5e2b-4055-9c86-0de7b82c782e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:16.664010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-35eba7e4-5e2b-4055-9c86-0de7b82c782e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:16.665231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-35eba7e4-5e2b-4055-9c86-0de7b82c782e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:16.773585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 665/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:16.828743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:16.829002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:16.829308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:16.830718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 649/1314] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jun 27 15:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:16.837186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab87d-1230-4c85-ac35-d852e4be0a12 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:16.837401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab87d-1230-4c85-ac35-d852e4be0a12 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:16.837673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ab87d-1230-4c85-ac35-d852e4be0a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:16.853165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:16.962903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-489ab87d-1230-4c85-ac35-d852e4be0a12 None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:16.964157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-489ab87d-1230-4c85-ac35-d852e4be0a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:17.054292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 666/1315] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jun 27 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:17.063849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:17.065600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:17.180409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab3a30-7dc4-4316-9353-66264d3bcc34 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:17.259606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:17.262880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:17.263666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:17.263955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:17.264339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:17.266778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:17.392574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:17.394040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f82222-3b2e-4d45-8e43-d10d6fc50ac0 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:17.413920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 650/1316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:20.075871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:20.076538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:20.076538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:20.077879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 667/1317] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:20.083961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b402a1-24aa-4977-831e-9bb3411f59f7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:20.084522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b402a1-24aa-4977-831e-9bb3411f59f7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:20.084522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b402a1-24aa-4977-831e-9bb3411f59f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:20.158423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-98b402a1-24aa-4977-831e-9bb3411f59f7 None None] MFA Rules not processed for user `0078685832a14fc8b0a4bbbb420b920b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:20.159586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-98b402a1-24aa-4977-831e-9bb3411f59f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:20.233637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:20.287991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 651/1318] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:20.328656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:20.330369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:20.454345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d573504-076e-462d-89db-b579cdc6e9fd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:20.457231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:20.535571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:20.536244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:20.536562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:20.536844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:20.538656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:20.659249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:20.660764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fd946-0498-47c2-a7b6-804337a85f8a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:20.683180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 668/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:23.546402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:23.547038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:23.547038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:23.548370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 652/1320] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:23.554224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b510-b993-4ead-9b67-902d016bee2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:23.554782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b510-b993-4ead-9b67-902d016bee2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:23.554782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6b510-b993-4ead-9b67-902d016bee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:23.597451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:23.627888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e3f6b510-b993-4ead-9b67-902d016bee2f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:23.629684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e3f6b510-b993-4ead-9b67-902d016bee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:23.738601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 669/1321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:23.778577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:23.780507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:23.897542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe51b2-dd70-40fe-ba77-0ac640aa5fba None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:23.975137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:23.976056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:23.976363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:23.976627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:23.978293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:23.986196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:24.104868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:24.106623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb113f1-8f82-499f-93b1-5f3bd3227078 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:24.126887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 653/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:27.036971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:27.037553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:27.037866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:27.040238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 670/1323] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:27.048025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2218d-3c95-4b4c-ab27-faddb156fee6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:27.048556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2218d-3c95-4b4c-ab27-faddb156fee6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:27.048756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2218d-3c95-4b4c-ab27-faddb156fee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:27.132184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-71b2218d-3c95-4b4c-ab27-faddb156fee6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:27.133342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-71b2218d-3c95-4b4c-ab27-faddb156fee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:27.198693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:27.240356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 654/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:27.275820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:27.277467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:27.397149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c4bc0-b258-43de-8e4a-93e14bc5efee None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:27.411568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:27.485470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:27.486154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:27.486434 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:27.486707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:27.488519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:27.615333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:27.616851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791ddd3-4d94-461e-b85c-e4b7875b61e8 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:27.637340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 671/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:29.845548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:29.846362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:29.846362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:29.847767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 655/1326] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:29.853412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdad6fd-444c-41e3-b675-81d2d43313a3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:29.854182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdad6fd-444c-41e3-b675-81d2d43313a3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:29.854182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdad6fd-444c-41e3-b675-81d2d43313a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:29.919552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:29.940994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9cdad6fd-444c-41e3-b675-81d2d43313a3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:29.942121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9cdad6fd-444c-41e3-b675-81d2d43313a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:30.059400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 672/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:18:30.097647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:30.099240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:30.244421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e9bd-da73-46be-9a44-cc615af8a7e7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:30.329008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:30.329676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:30.329961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:30.330240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:30.331938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:30.351331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:30.459056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:30.460634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76936fa1-79ee-4f71-9b0a-c2193519d4cb service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:30.494580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 656/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:32.723879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:32.724580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:32.724950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:32.727326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 673/1329] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:32.736769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87088-b606-48fd-8de1-6f7057c9b77a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:32.738064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87088-b606-48fd-8de1-6f7057c9b77a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:32.738064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa87088-b606-48fd-8de1-6f7057c9b77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:32.839896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1aa87088-b606-48fd-8de1-6f7057c9b77a None None] MFA Rules not processed for user `0078685832a14fc8b0a4bbbb420b920b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:32.841572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1aa87088-b606-48fd-8de1-6f7057c9b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:32.928965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:32.968580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 657/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:33.008360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:33.010154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:33.129316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a0219-acb9-48d0-b62f-a54005857902 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:33.158555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:33.236280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:33.237436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:33.237900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:33.238260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:33.240144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:33.365886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:33.367072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dc823-9dcb-48fe-9357-29a369b87474 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:33.387333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 674/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:35.861321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:35.861872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:35.861872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:35.863309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 658/1332] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:35.869682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c10b-756a-407e-9153-103c7b5eba50 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:35.870224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c10b-756a-407e-9153-103c7b5eba50 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:35.870224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c10b-756a-407e-9153-103c7b5eba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:35.942462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:35.945802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8ca7c10b-756a-407e-9153-103c7b5eba50 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:35.947996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8ca7c10b-756a-407e-9153-103c7b5eba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:36.066146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 675/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:36.109266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:36.111474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:36.233232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c73960-a42f-4dec-a86e-95cdbe192c5b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:36.350136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:36.351295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:36.351817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:36.352257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:36.355433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:36.398612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:36.527145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:36.528875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fc31b-7824-484e-9a44-9012414f128a service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:36.550900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 659/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:38.734934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:38.735633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:38.735633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:38.736971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 676/1335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:38.742510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b6ac-b9c0-464d-a797-7d5df695521d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:38.743134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b6ac-b9c0-464d-a797-7d5df695521d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:38.743134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0b6ac-b9c0-464d-a797-7d5df695521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:38.814518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3cb0b6ac-b9c0-464d-a797-7d5df695521d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:38.815821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3cb0b6ac-b9c0-464d-a797-7d5df695521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:38.895388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:38.923739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 660/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:38.972562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:38.974120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:39.099233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c10cb9-99a3-4fe1-b576-761186c2257f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:39.132772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:39.188741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:39.189403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:39.189688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:39.189948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:39.191657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:39.316924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:39.318763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407ac0-f522-4bdb-a3bd-3c8c0aaa01a5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:39.338922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 677/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:41.875095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976832-b069-4a52-939b-cfad47037151 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:41.875719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976832-b069-4a52-939b-cfad47037151 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:41.876220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976832-b069-4a52-939b-cfad47037151 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:41.879776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 661/1338] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:41.885864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d1d20-7292-4ce5-8e82-a72137b6a80a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:41.886420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d1d20-7292-4ce5-8e82-a72137b6a80a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:41.886420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d1d20-7292-4ce5-8e82-a72137b6a80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:41.968120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-944d1d20-7292-4ce5-8e82-a72137b6a80a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:41.969518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-944d1d20-7292-4ce5-8e82-a72137b6a80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:41.979773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:42.087526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 678/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:18:42.126336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c976832-b069-4a52-939b-cfad47037151 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:42.127911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9c976832-b069-4a52-939b-cfad47037151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:42.252550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c976832-b069-4a52-939b-cfad47037151 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:42.337223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:42.337888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:42.338168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:42.338439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:42.340096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:42.382204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:42.460117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:42.461616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6d405-7377-4295-80de-c083020ce149 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:42.481021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 662/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:46.784244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fe010-c277-496d-945a-b14c7650b66f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:46.785475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fe010-c277-496d-945a-b14c7650b66f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:46.785475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fe010-c277-496d-945a-b14c7650b66f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:46.787337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 679/1341] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:46.795515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15fcd-37f7-48dd-b1d6-687b9b8c9eb7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:46.795982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15fcd-37f7-48dd-b1d6-687b9b8c9eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:46.796083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15fcd-37f7-48dd-b1d6-687b9b8c9eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:46.904662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-77f15fcd-37f7-48dd-b1d6-687b9b8c9eb7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:46.905794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77f15fcd-37f7-48dd-b1d6-687b9b8c9eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:47.015987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:47.032270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 663/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:47.072778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fe010-c277-496d-945a-b14c7650b66f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:47.075605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-422fe010-c277-496d-945a-b14c7650b66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:47.218376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fe010-c277-496d-945a-b14c7650b66f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:47.222142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:47.222801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:47.223112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:47.223380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:47.234534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:47.236034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4c5db-f856-4668-9ecc-d1dfd55675f0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:47.248404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 680/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:50.091350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f103-75cf-4355-8815-77d58e71ce9b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:50.091919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f103-75cf-4355-8815-77d58e71ce9b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:50.092335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7f103-75cf-4355-8815-77d58e71ce9b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:50.093788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 664/1344] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:18:50.099512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cef3d-2078-4ed7-8b4f-f78e623c051a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:50.101137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cef3d-2078-4ed7-8b4f-f78e623c051a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:50.101137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cef3d-2078-4ed7-8b4f-f78e623c051a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:50.130382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-af5cef3d-2078-4ed7-8b4f-f78e623c051a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:50.134417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-af5cef3d-2078-4ed7-8b4f-f78e623c051a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:50.164769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:18:50.239303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 681/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:18:50.306214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7f103-75cf-4355-8815-77d58e71ce9b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:50.322087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7f103-75cf-4355-8815-77d58e71ce9b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:50.326283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:50.326949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:50.327233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:50.327512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:50.337546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:50.339082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17feef29-bf12-48ea-bdc8-b29bce0119ca service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:50.350779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 665/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:18:54.875014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41acb-6792-47c3-8649-f0b418a3c598 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:54.875631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41acb-6792-47c3-8649-f0b418a3c598 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:54.875631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d41acb-6792-47c3-8649-f0b418a3c598 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:54.877281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 682/1347] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:18:54.882877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79e6c-8d03-466a-b22c-0d3c87290026 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:54.883419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79e6c-8d03-466a-b22c-0d3c87290026 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:54.883628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79e6c-8d03-466a-b22c-0d3c87290026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:54.903583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bcf79e6c-8d03-466a-b22c-0d3c87290026 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:18:54.904871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bcf79e6c-8d03-466a-b22c-0d3c87290026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:18:54.913689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 666/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:18:54.955969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41acb-6792-47c3-8649-f0b418a3c598 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:18:54.969449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d41acb-6792-47c3-8649-f0b418a3c598 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:18:54.973674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:18:54.974481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:18:54.974888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:18:54.975282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:18:54.987053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:18:54.988698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c758a87-07ad-4133-add9-9376ef0c5af9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:18:54.999191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 683/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:00.238630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6d8b-4d4f-4b59-a94f-2605d4d02b78 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:00.239478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6d8b-4d4f-4b59-a94f-2605d4d02b78 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:00.239902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6d8b-4d4f-4b59-a94f-2605d4d02b78 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:00.242647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 667/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:19:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:19:00.258536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2fcb-1896-41e8-819d-e5aa12c1098f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:00.259410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2fcb-1896-41e8-819d-e5aa12c1098f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:00.259837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2fcb-1896-41e8-819d-e5aa12c1098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:00.387487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2a2c2fcb-1896-41e8-819d-e5aa12c1098f None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:00.388891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2a2c2fcb-1896-41e8-819d-e5aa12c1098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:00.478501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 684/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:04.809569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ae36-059a-4915-813d-51deb921ca0d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:04.811297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ae36-059a-4915-813d-51deb921ca0d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:04.811800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ae36-059a-4915-813d-51deb921ca0d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:04.813612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 668/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:19:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:04.819688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ad4-3e0f-41aa-80f1-3261c8efe3d3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:04.820495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ad4-3e0f-41aa-80f1-3261c8efe3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:04.820495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e106ad4-3e0f-41aa-80f1-3261c8efe3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:04.840489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8e106ad4-3e0f-41aa-80f1-3261c8efe3d3 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:04.841777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8e106ad4-3e0f-41aa-80f1-3261c8efe3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:04.854022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 685/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:04.896611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=13 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:06.326805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ddf7d-69ed-49b4-8445-f34f712016bf None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:06.327465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ddf7d-69ed-49b4-8445-f34f712016bf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:06.327465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ddf7d-69ed-49b4-8445-f34f712016bf None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:06.329911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 669/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:06.335598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c71c-67b5-433c-abc9-d66e32c2a0a5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:06.336181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c71c-67b5-433c-abc9-d66e32c2a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:06.336181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c71c-67b5-433c-abc9-d66e32c2a0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:06.357632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2574c71c-67b5-433c-abc9-d66e32c2a0a5 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:06.358992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2574c71c-67b5-433c-abc9-d66e32c2a0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:06.368116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 686/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:10.072122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef72d-7570-415b-9072-2a0dd6dfa4dc None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:10.072745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef72d-7570-415b-9072-2a0dd6dfa4dc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:10.078941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ef72d-7570-415b-9072-2a0dd6dfa4dc None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:10.080770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 670/1356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:10.086857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3f96-63aa-472a-88ad-0b1ea2efb6a6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:10.087814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3f96-63aa-472a-88ad-0b1ea2efb6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:10.089032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3f96-63aa-472a-88ad-0b1ea2efb6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:10.112689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-24cc3f96-63aa-472a-88ad-0b1ea2efb6a6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:10.114329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-24cc3f96-63aa-472a-88ad-0b1ea2efb6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:10.123435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 687/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:10.192676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcaadf-138c-4890-987f-675397238a30 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:10.193248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcaadf-138c-4890-987f-675397238a30 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:10.193622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcaadf-138c-4890-987f-675397238a30 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:10.195479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 671/1358] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jun 27 15:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:10.209820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb727ff-9aba-4d7d-b4d7-b2e481cf34df None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:10.210111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb727ff-9aba-4d7d-b4d7-b2e481cf34df None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:10.210391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb727ff-9aba-4d7d-b4d7-b2e481cf34df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:10.258555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:10.268287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9fb727ff-9aba-4d7d-b4d7-b2e481cf34df None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:10.269536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9fb727ff-9aba-4d7d-b4d7-b2e481cf34df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:10.360157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 688/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jun 27 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:10.368247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcaadf-138c-4890-987f-675397238a30 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:10.383503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcaadf-138c-4890-987f-675397238a30 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:10.393276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:10.394238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:10.394873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:10.395850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:10.407950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:10.409974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee118e-8471-4ba5-97f7-e6b5affd2d30 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:10.437436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 672/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:12.688688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159891a4-675e-44be-a6a4-cdf0fac9b15d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:12.689593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159891a4-675e-44be-a6a4-cdf0fac9b15d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:12.689980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159891a4-675e-44be-a6a4-cdf0fac9b15d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:12.693005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 689/1361] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:12.697199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a6cf-8cbd-4de6-8e85-155b5849f2af None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:12.698023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a6cf-8cbd-4de6-8e85-155b5849f2af None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:12.698305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a6cf-8cbd-4de6-8e85-155b5849f2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:12.718801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-af76a6cf-8cbd-4de6-8e85-155b5849f2af None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:12.720018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-af76a6cf-8cbd-4de6-8e85-155b5849f2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:12.728208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 673/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:12.778550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159891a4-675e-44be-a6a4-cdf0fac9b15d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:12.794347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159891a4-675e-44be-a6a4-cdf0fac9b15d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:12.799921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:12.800652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:12.800935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:12.801209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:12.810909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:12.812560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e8148-9f6e-4b14-9f46-27f42fce47fe service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:12.822880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 690/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:15.207193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d3a3-3811-4bac-b70e-ee6f51ed9b19 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:15.207698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d3a3-3811-4bac-b70e-ee6f51ed9b19 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:15.208182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09d3a3-3811-4bac-b70e-ee6f51ed9b19 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:15.210066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 674/1364] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:15.217094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bd05-656a-44f3-84b9-039f80ad02b9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:15.217861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bd05-656a-44f3-84b9-039f80ad02b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:15.217861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bd05-656a-44f3-84b9-039f80ad02b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:15.239009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0028bd05-656a-44f3-84b9-039f80ad02b9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:15.240307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0028bd05-656a-44f3-84b9-039f80ad02b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:15.250751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 691/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:15.301058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22dca-d414-411a-accc-57c8dae1ab76 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:15.301413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22dca-d414-411a-accc-57c8dae1ab76 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:15.301757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c22dca-d414-411a-accc-57c8dae1ab76 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:15.303202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 675/1366] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jun 27 15:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:15.308948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1fb0e-2faa-47d8-8a05-ace34be8ff2a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:15.309232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1fb0e-2faa-47d8-8a05-ace34be8ff2a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:15.309516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1fb0e-2faa-47d8-8a05-ace34be8ff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:15.329153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9ad1fb0e-2faa-47d8-8a05-ace34be8ff2a None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:15.330283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9ad1fb0e-2faa-47d8-8a05-ace34be8ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:15.339753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 692/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jun 27 15:19:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:15.348433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22dca-d414-411a-accc-57c8dae1ab76 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:15.363954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c22dca-d414-411a-accc-57c8dae1ab76 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:15.371216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:15.371216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:15.371216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:15.371216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:15.380420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:15.382261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55001a48-5ff5-4be8-969c-72bc0eb48318 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:15.398088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 676/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:17.530576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adfdd-ce19-4377-a368-ae03db749d28 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:17.531170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adfdd-ce19-4377-a368-ae03db749d28 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:17.531379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36adfdd-ce19-4377-a368-ae03db749d28 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:17.532880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 693/1369] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:17.538877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9c53-221f-49cd-97de-dbaf48eea830 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:17.539476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9c53-221f-49cd-97de-dbaf48eea830 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:17.539476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb9c53-221f-49cd-97de-dbaf48eea830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:17.559905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a2cb9c53-221f-49cd-97de-dbaf48eea830 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:17.561147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a2cb9c53-221f-49cd-97de-dbaf48eea830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:17.572374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 677/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:17.615456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36adfdd-ce19-4377-a368-ae03db749d28 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:17.631568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36adfdd-ce19-4377-a368-ae03db749d28 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:17.637576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:17.638471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:17.639000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:17.639476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:17.648837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:17.650733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42dea6-750d-43a7-884f-a073deaadaa2 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:17.661587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 694/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:18.440735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f61-d522-49e9-bba0-de5944ebe769 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:18.441767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f61-d522-49e9-bba0-de5944ebe769 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:18.442080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e3f61-d522-49e9-bba0-de5944ebe769 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:18.443661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 678/1372] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:18.452720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9a5b-e4aa-4840-8dee-9ce0b95ad08f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:18.453012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9a5b-e4aa-4840-8dee-9ce0b95ad08f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:18.453291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9a5b-e4aa-4840-8dee-9ce0b95ad08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:18.474125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b12e9a5b-e4aa-4840-8dee-9ce0b95ad08f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:18.475518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b12e9a5b-e4aa-4840-8dee-9ce0b95ad08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:18.490694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 695/1373] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:18.843784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19d9f-774a-47da-b206-a1fbc7ff0b5c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:18.843784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19d9f-774a-47da-b206-a1fbc7ff0b5c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:18.843784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf19d9f-774a-47da-b206-a1fbc7ff0b5c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:18.846600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 679/1374] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jun 27 15:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:18.850856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff254-8ec8-4c7e-84e3-2e7bca13adc6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:18.851466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff254-8ec8-4c7e-84e3-2e7bca13adc6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:18.851466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ff254-8ec8-4c7e-84e3-2e7bca13adc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:18.906581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cb7ff254-8ec8-4c7e-84e3-2e7bca13adc6 None None] MFA Rules not processed for user `4b9a1a77fd5b40f6b531c79d69044430`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:18.907953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb7ff254-8ec8-4c7e-84e3-2e7bca13adc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:18.976387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 696/1375] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jun 27 15:19:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:18.984253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf19d9f-774a-47da-b206-a1fbc7ff0b5c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:18.996032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf19d9f-774a-47da-b206-a1fbc7ff0b5c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:19.000101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:19.000882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:19.001238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:19.001607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:19.012117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:19.013719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48429a73-d3f7-4ae1-b071-81d87adf0840 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:19.024530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 680/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:19.778017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b7e3-4c3a-4792-b1c1-760a61663a3e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:19.778309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b7e3-4c3a-4792-b1c1-760a61663a3e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:19.778876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b7e3-4c3a-4792-b1c1-760a61663a3e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:19.780616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 697/1377] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:19.786025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110065-6e21-44fb-9c25-a227d3dd4e64 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:19.786205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110065-6e21-44fb-9c25-a227d3dd4e64 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:19.786790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110065-6e21-44fb-9c25-a227d3dd4e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:19.805715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-66110065-6e21-44fb-9c25-a227d3dd4e64 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:19.806850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66110065-6e21-44fb-9c25-a227d3dd4e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:19.816799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 681/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:19:19.860790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2b7e3-4c3a-4792-b1c1-760a61663a3e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:19.884646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2b7e3-4c3a-4792-b1c1-760a61663a3e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:19.891272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:19.891940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:19.892225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:19.892545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:19.902153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:19.903915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5a7bc-c389-4ce3-9dd4-2c0500ec8c0a service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:19.908467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:19.924332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 698/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:22.048380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c630-c85e-432c-b7fd-cef4da4400e1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:22.048980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c630-c85e-432c-b7fd-cef4da4400e1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:22.048980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4c630-c85e-432c-b7fd-cef4da4400e1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:22.050657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 682/1380] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:22.056565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36661a-2e17-46d9-a873-cfb0b66bf226 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:22.057027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36661a-2e17-46d9-a873-cfb0b66bf226 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:22.057233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36661a-2e17-46d9-a873-cfb0b66bf226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:22.090845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1a36661a-2e17-46d9-a873-cfb0b66bf226 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:22.092066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1a36661a-2e17-46d9-a873-cfb0b66bf226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:22.100442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 699/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:22.145190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c630-c85e-432c-b7fd-cef4da4400e1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:22.160998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4c630-c85e-432c-b7fd-cef4da4400e1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:22.166444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:22.166444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:22.168703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:22.168703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:22.177950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:22.179569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd2fed-4e8b-41d6-8aa1-5d32754dc17c service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:22.190362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 683/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:24.314508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d4da-d602-4e6e-9572-090ee020c08e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:24.315155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d4da-d602-4e6e-9572-090ee020c08e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:24.315155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d4da-d602-4e6e-9572-090ee020c08e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:24.316805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 700/1383] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:24.322333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560641-0b58-4258-83a3-ada96cf954f1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:24.322951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560641-0b58-4258-83a3-ada96cf954f1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:24.322951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560641-0b58-4258-83a3-ada96cf954f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:24.342597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f1560641-0b58-4258-83a3-ada96cf954f1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:24.345138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f1560641-0b58-4258-83a3-ada96cf954f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:24.353661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 684/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:24.395701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d4da-d602-4e6e-9572-090ee020c08e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:24.412416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d4da-d602-4e6e-9572-090ee020c08e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:24.419897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:24.420933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:24.421394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:24.421962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:24.432174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:24.434102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996e62-4342-457a-9b6a-5f1d7a820ca5 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:24.447357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 701/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:26.532102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f427d-6eea-444c-8e55-1367994a457b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:26.532724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f427d-6eea-444c-8e55-1367994a457b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:26.532724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f427d-6eea-444c-8e55-1367994a457b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:26.534070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 685/1386] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:26.539776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ebe6-fe8f-410d-b094-9a7997b81c80 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:26.540262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ebe6-fe8f-410d-b094-9a7997b81c80 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:26.540393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2ebe6-fe8f-410d-b094-9a7997b81c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:26.560388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-67c2ebe6-fe8f-410d-b094-9a7997b81c80 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:26.561800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-67c2ebe6-fe8f-410d-b094-9a7997b81c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:26.571825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 702/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:26.617326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f427d-6eea-444c-8e55-1367994a457b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:26.633475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f427d-6eea-444c-8e55-1367994a457b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:26.639415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:26.640046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:26.640484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:26.640581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:26.649241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:26.650844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53834ad2-8577-4cac-abdc-685555fac31e service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:26.672568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 686/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:29.137846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad890530-f321-45f3-92a4-bf4024459beb None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:29.138638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad890530-f321-45f3-92a4-bf4024459beb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:29.138823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad890530-f321-45f3-92a4-bf4024459beb None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:29.140664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 703/1389] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:29.147329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376272c8-a153-4cec-8f27-7d672d476f43 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:29.148241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376272c8-a153-4cec-8f27-7d672d476f43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:29.148241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376272c8-a153-4cec-8f27-7d672d476f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:29.168890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-376272c8-a153-4cec-8f27-7d672d476f43 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:29.170150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-376272c8-a153-4cec-8f27-7d672d476f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:29.180704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 687/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:29.236399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad890530-f321-45f3-92a4-bf4024459beb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:29.253880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad890530-f321-45f3-92a4-bf4024459beb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:29.259894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:29.261031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:29.261465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:29.261883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:29.273547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:29.275213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd147876-8349-41f1-baa2-9de3497b6c21 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:29.287224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 704/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:32.093946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c076-fd76-4ad6-89d4-0894749ddfad None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:32.094522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c076-fd76-4ad6-89d4-0894749ddfad None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:32.094522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407c076-fd76-4ad6-89d4-0894749ddfad None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:32.096186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 688/1392] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:32.102857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c61f-f691-420e-8a00-9fb9f225ff0a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:32.103363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c61f-f691-420e-8a00-9fb9f225ff0a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:32.103640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c61f-f691-420e-8a00-9fb9f225ff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:32.129562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1ea2c61f-f691-420e-8a00-9fb9f225ff0a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:32.131139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ea2c61f-f691-420e-8a00-9fb9f225ff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:32.141959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 705/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:19:32.201656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407c076-fd76-4ad6-89d4-0894749ddfad None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:32.220921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407c076-fd76-4ad6-89d4-0894749ddfad None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:32.228339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:32.229376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:32.229696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:32.230114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:32.242404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:32.244488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d5a3f-eb93-4197-a212-ce5069db20a5 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:32.256803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 689/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:34.532458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bb37-a313-48d3-b615-d68ece230f5e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.533115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bb37-a313-48d3-b615-d68ece230f5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.533115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bb37-a313-48d3-b615-d68ece230f5e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.534398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 706/1395] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:34.540182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0698b-0483-4525-808c-fb7105cea8b7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.540825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0698b-0483-4525-808c-fb7105cea8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.540825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0698b-0483-4525-808c-fb7105cea8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.561229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-33e0698b-0483-4525-808c-fb7105cea8b7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:34.562461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-33e0698b-0483-4525-808c-fb7105cea8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:34.571616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 690/1396] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:34.616778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843bb37-a313-48d3-b615-d68ece230f5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:34.632871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843bb37-a313-48d3-b615-d68ece230f5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:34.639729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:34.640414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.640701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.640965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.650803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:34.652650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e3655-d772-4153-abdd-d1cb897db459 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:34.663800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 707/1397] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:34.811191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494680a-336e-4088-8621-218dec1a4c73 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.811491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494680a-336e-4088-8621-218dec1a4c73 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.811724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494680a-336e-4088-8621-218dec1a4c73 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.813251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 691/1398] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:34.820102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685332bd-b595-458e-9e99-619d8b9fec4c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.820401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685332bd-b595-458e-9e99-619d8b9fec4c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.820700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685332bd-b595-458e-9e99-619d8b9fec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.840055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-685332bd-b595-458e-9e99-619d8b9fec4c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:34.841332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-685332bd-b595-458e-9e99-619d8b9fec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:34.852082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 708/1399] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:34.909150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651375b-c0f7-436c-a8ba-d844e7214aa9 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.909372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651375b-c0f7-436c-a8ba-d844e7214aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.909645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651375b-c0f7-436c-a8ba-d844e7214aa9 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.911038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 692/1400] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jun 27 15:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:34.916540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d43ce-3086-4435-aa61-d8f06b476a19 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:34.916956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d43ce-3086-4435-aa61-d8f06b476a19 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:34.917252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d43ce-3086-4435-aa61-d8f06b476a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:34.968636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-161d43ce-3086-4435-aa61-d8f06b476a19 None None] MFA Rules not processed for user `62ae50de406e46dbb2448a8f04426bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:34.969838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-161d43ce-3086-4435-aa61-d8f06b476a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:35.033668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 709/1401] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jun 27 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:35.043072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651375b-c0f7-436c-a8ba-d844e7214aa9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:35.049182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=55 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:35.056334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651375b-c0f7-436c-a8ba-d844e7214aa9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:35.060903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:35.061571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:35.061846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:35.062132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:35.073479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:35.075062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c596ac-301b-4f89-9665-806e97802796 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:35.086847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 693/1402] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:35.852354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc9ec-5f92-49fb-84cf-4d08a348493a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:35.852917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc9ec-5f92-49fb-84cf-4d08a348493a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:35.852917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cc9ec-5f92-49fb-84cf-4d08a348493a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:35.908787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0a8cc9ec-5f92-49fb-84cf-4d08a348493a None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:35.910000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0a8cc9ec-5f92-49fb-84cf-4d08a348493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:35.953446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:36.001512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 710/1403] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:36.052758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd10-3c31-4d0c-b6c1-f7c152a01a52 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.053246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd10-3c31-4d0c-b6c1-f7c152a01a52 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.053337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd10-3c31-4d0c-b6c1-f7c152a01a52 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.054705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 694/1404] 10.222.0.26 () {54 vars in 942 bytes} [Mon Jun 27 15:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:36.062277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915a0c-1982-4420-b1be-f3504789af23 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.062561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915a0c-1982-4420-b1be-f3504789af23 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.062841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73915a0c-1982-4420-b1be-f3504789af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.116593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-73915a0c-1982-4420-b1be-f3504789af23 None None] MFA Rules not processed for user `47acb0dcc4164628a2442c7dd26d478b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:36.117797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-73915a0c-1982-4420-b1be-f3504789af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:36.195190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 711/1405] 10.222.0.26 () {58 vars in 1039 bytes} [Mon Jun 27 15:19:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:19:36.205373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286bd10-3c31-4d0c-b6c1-f7c152a01a52 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:36.220751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286bd10-3c31-4d0c-b6c1-f7c152a01a52 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:36.227239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:36.227906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.228219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.228500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.237861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:36.239562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aeffdc-56c0-42d4-8cd7-863bef8fd616 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:36.248216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 695/1406] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:19:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:19:36.774836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c79081-f542-4ab0-b9f7-0d43e2b4c51f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.775200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c79081-f542-4ab0-b9f7-0d43e2b4c51f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.775410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c79081-f542-4ab0-b9f7-0d43e2b4c51f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.782626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcdd55-8f1c-47d4-aecb-15995db51fac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.782922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcdd55-8f1c-47d4-aecb-15995db51fac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.783205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcdd55-8f1c-47d4-aecb-15995db51fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.784482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 712/1407] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:19:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:36.801499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-88fcdd55-8f1c-47d4-aecb-15995db51fac None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:36.806849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-88fcdd55-8f1c-47d4-aecb-15995db51fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:36.815380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 696/1408] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:36.862899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c79081-f542-4ab0-b9f7-0d43e2b4c51f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:19:36.879616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c79081-f542-4ab0-b9f7-0d43e2b4c51f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:19:36.885133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:19:36.886019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:36.886402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:36.886652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:36.895588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:19:36.897240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c690571-103e-43bc-b633-7b8d4cf82be5 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:19:36.907267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 713/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:19:46.796216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=14 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:46.902683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=59 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:19:58.888489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f28718-0666-4dd0-b12c-5daee61a3919 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:58.889051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f28718-0666-4dd0-b12c-5daee61a3919 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:58.889051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f28718-0666-4dd0-b12c-5daee61a3919 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:58.890485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 697/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jun 27 15:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:58.896006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4be1-1b31-4e76-9eeb-abf720408032 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:58.896477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4be1-1b31-4e76-9eeb-abf720408032 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:58.896572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe4be1-1b31-4e76-9eeb-abf720408032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:58.916093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-45fe4be1-1b31-4e76-9eeb-abf720408032 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:58.917493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-45fe4be1-1b31-4e76-9eeb-abf720408032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:58.927019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 714/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jun 27 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:19:58.942778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755dee1-c05f-497e-b291-5ed36ec0b10c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:58.943061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755dee1-c05f-497e-b291-5ed36ec0b10c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:58.943334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755dee1-c05f-497e-b291-5ed36ec0b10c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:58.944738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 698/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jun 27 15:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:19:58.950782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ac60-aa80-4f6f-acfd-49864b1adbd1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:19:58.951171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ac60-aa80-4f6f-acfd-49864b1adbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:19:58.951554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3ac60-aa80-4f6f-acfd-49864b1adbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:19:58.970617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3ab3ac60-aa80-4f6f-acfd-49864b1adbd1 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:19:58.971822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3ab3ac60-aa80-4f6f-acfd-49864b1adbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:19:58.981797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 715/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jun 27 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:01.544502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cf7b-329a-4bba-9be6-8846265ad3fa None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:01.545128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cf7b-329a-4bba-9be6-8846265ad3fa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:01.545128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972cf7b-329a-4bba-9be6-8846265ad3fa None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:01.546426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:01.552014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d147a7c-aee3-4d3a-9fc0-dcf502931e62 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:01.552512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d147a7c-aee3-4d3a-9fc0-dcf502931e62 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:01.552619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d147a7c-aee3-4d3a-9fc0-dcf502931e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:01.572162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4d147a7c-aee3-4d3a-9fc0-dcf502931e62 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:01.574061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4d147a7c-aee3-4d3a-9fc0-dcf502931e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:01.582288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:01.626839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972cf7b-329a-4bba-9be6-8846265ad3fa None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:01.642485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972cf7b-329a-4bba-9be6-8846265ad3fa None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:01.648007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:01.648691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:01.648972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:01.649246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:01.658544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:01.660131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932acfbe-f72a-4b67-9c10-3b03edbf0358 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:01.670622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 700/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:02.617227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da950-c523-46a7-a349-3e48f38e8713 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:02.617808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da950-c523-46a7-a349-3e48f38e8713 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:02.617808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da950-c523-46a7-a349-3e48f38e8713 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:02.619171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:02.624682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d43fd-e91d-4a2a-89d5-b98182366dd3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:02.625207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d43fd-e91d-4a2a-89d5-b98182366dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:02.625207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d43fd-e91d-4a2a-89d5-b98182366dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:02.644515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8f6d43fd-e91d-4a2a-89d5-b98182366dd3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:02.645756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8f6d43fd-e91d-4a2a-89d5-b98182366dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:02.654316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:02.812883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785da950-c523-46a7-a349-3e48f38e8713 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:02.830111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785da950-c523-46a7-a349-3e48f38e8713 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:02.836468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:02.837119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:02.837408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:02.837682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:02.847617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:02.849272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af911e93-68c4-428b-ba6a-ef3282a15f7f service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:02.859260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 718/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:03.761015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1703448-6763-47e7-944a-1675f0a30af1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:03.761597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1703448-6763-47e7-944a-1675f0a30af1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:03.761597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1703448-6763-47e7-944a-1675f0a30af1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:03.762941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:03.768599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec90f2-ffcd-494c-9758-76fd78fe5b1d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:03.769049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec90f2-ffcd-494c-9758-76fd78fe5b1d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:03.769194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec90f2-ffcd-494c-9758-76fd78fe5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:03.788842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e0ec90f2-ffcd-494c-9758-76fd78fe5b1d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:03.790073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0ec90f2-ffcd-494c-9758-76fd78fe5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:03.799913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:03.948778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1703448-6763-47e7-944a-1675f0a30af1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:03.965770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1703448-6763-47e7-944a-1675f0a30af1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:03.971395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:03.972051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:03.972362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:03.972672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:03.981917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:03.983534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aab76-4154-424d-b924-f42ed2ea113d service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:03.993402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 703/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:04.863751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173b3d-a08d-4e98-9286-fd0e82f7640a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:04.864419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173b3d-a08d-4e98-9286-fd0e82f7640a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:04.864419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173b3d-a08d-4e98-9286-fd0e82f7640a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:04.865731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:04.871737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb2f-9f4d-4e99-b961-3cfb1cf61f18 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:04.872258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb2f-9f4d-4e99-b961-3cfb1cf61f18 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:04.872258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabeb2f-9f4d-4e99-b961-3cfb1cf61f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:04.891737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bcabeb2f-9f4d-4e99-b961-3cfb1cf61f18 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:04.893098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bcabeb2f-9f4d-4e99-b961-3cfb1cf61f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:04.902656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:05.053351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e173b3d-a08d-4e98-9286-fd0e82f7640a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:05.069852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e173b3d-a08d-4e98-9286-fd0e82f7640a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:05.076207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:05.076888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:05.077169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:05.077447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:05.087091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:05.089217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4be7c-2c52-4885-89ac-7c76794e6350 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:05.099406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 721/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:05.985594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afcb5-89c0-4b4f-834f-89894d67aeea None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:05.986190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afcb5-89c0-4b4f-834f-89894d67aeea None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:05.986190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afcb5-89c0-4b4f-834f-89894d67aeea None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:05.987526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:05.992748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d062-6db3-4430-a0f6-542cdd151dd0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:05.993221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d062-6db3-4430-a0f6-542cdd151dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:05.993324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d062-6db3-4430-a0f6-542cdd151dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:06.017474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6708d062-6db3-4430-a0f6-542cdd151dd0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:06.018710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6708d062-6db3-4430-a0f6-542cdd151dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:06.029622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:06.181391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6afcb5-89c0-4b4f-834f-89894d67aeea None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:06.197729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6afcb5-89c0-4b4f-834f-89894d67aeea None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:06.203506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:06.204139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:06.204451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:06.204732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:06.213809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:06.215493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76062d27-fc3c-42d9-8de3-51ea8a9f2e72 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:06.225316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 706/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:07.072177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1fbdf-84fa-48f4-9ae6-80f0eaf37add None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:07.072851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1fbdf-84fa-48f4-9ae6-80f0eaf37add None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:07.073058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1fbdf-84fa-48f4-9ae6-80f0eaf37add None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:07.074851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:07.080096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578251-c7c5-430f-ac57-9e5ebd18b61b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:07.080614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578251-c7c5-430f-ac57-9e5ebd18b61b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:07.080729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578251-c7c5-430f-ac57-9e5ebd18b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:07.100870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fc578251-c7c5-430f-ac57-9e5ebd18b61b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:07.102109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fc578251-c7c5-430f-ac57-9e5ebd18b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:07.111725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:07.263546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1fbdf-84fa-48f4-9ae6-80f0eaf37add None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:07.279384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1fbdf-84fa-48f4-9ae6-80f0eaf37add None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:07.285486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:07.286144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:07.286427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:07.286704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:07.296793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:07.298418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf5d8d-a0dd-4814-901d-62c920e6e594 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:07.308690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 724/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:08.168117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87a3d-340b-4590-9565-8bae4e50e5ab None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:08.168839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87a3d-340b-4590-9565-8bae4e50e5ab None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:08.168839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87a3d-340b-4590-9565-8bae4e50e5ab None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:08.170012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:08.175369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3dd8-8805-4697-b7ea-135d2e3896b5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:08.175871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3dd8-8805-4697-b7ea-135d2e3896b5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:08.175871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3dd8-8805-4697-b7ea-135d2e3896b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:08.196969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-789a3dd8-8805-4697-b7ea-135d2e3896b5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:08.198835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-789a3dd8-8805-4697-b7ea-135d2e3896b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:08.207672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:08.360226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec87a3d-340b-4590-9565-8bae4e50e5ab None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:08.376830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec87a3d-340b-4590-9565-8bae4e50e5ab None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:08.382798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:08.383448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:08.383735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:08.384011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:08.392914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:08.394520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa46ef4-bc1e-466e-9dde-34a3e3407d44 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:08.404413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 709/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:09.209859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81b9-71fa-4fab-acc0-e1ed25e392f1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:09.210479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81b9-71fa-4fab-acc0-e1ed25e392f1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:09.210479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81b9-71fa-4fab-acc0-e1ed25e392f1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:09.211840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:09.217951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ca6ae-a019-4410-987e-e81d6874cd62 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:09.218559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ca6ae-a019-4410-987e-e81d6874cd62 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:09.218559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ca6ae-a019-4410-987e-e81d6874cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:09.238257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-576ca6ae-a019-4410-987e-e81d6874cd62 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:09.239494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-576ca6ae-a019-4410-987e-e81d6874cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:09.248126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:09.398287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e81b9-71fa-4fab-acc0-e1ed25e392f1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:09.415316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e81b9-71fa-4fab-acc0-e1ed25e392f1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:09.421970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:09.422629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:09.422919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:09.423199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:09.432251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:09.433971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994290-3d7a-4030-b0b3-3d4afe248a0d service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:09.444574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 727/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:10.272885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57780be9-cce2-4d41-82fa-0f40338a0b6e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:10.273515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57780be9-cce2-4d41-82fa-0f40338a0b6e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:10.273515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57780be9-cce2-4d41-82fa-0f40338a0b6e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:10.275031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:10.280939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d5902-1259-4587-ae5f-305bf8524b23 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:10.281508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d5902-1259-4587-ae5f-305bf8524b23 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:10.281508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d5902-1259-4587-ae5f-305bf8524b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:10.300233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7e7d5902-1259-4587-ae5f-305bf8524b23 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:10.301529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7e7d5902-1259-4587-ae5f-305bf8524b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:10.311050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:10.460959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57780be9-cce2-4d41-82fa-0f40338a0b6e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:10.477289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57780be9-cce2-4d41-82fa-0f40338a0b6e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:10.484607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:10.485253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:10.485548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:10.485825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:10.494774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:10.496389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a7701-7a7b-4164-92fb-22dacf4d2923 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:10.505957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 712/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:12.427567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:12.428148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:12.428148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:12.429552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:12.435671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1acf-e2f9-4cb6-be77-3884649bceda None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:12.436102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1acf-e2f9-4cb6-be77-3884649bceda None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:12.436232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c1acf-e2f9-4cb6-be77-3884649bceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:12.456699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-963c1acf-e2f9-4cb6-be77-3884649bceda None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:12.457932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-963c1acf-e2f9-4cb6-be77-3884649bceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:12.467367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:12.506384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:12.508295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:12.524268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78b1f8-3a71-4ef6-ba20-c00e2c7371f8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:12.528853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:12.529488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:12.529764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:12.530038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:12.539959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:12.541636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a2d49d-08e0-463c-a6d4-b6a3a261bbee service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:12.552080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 730/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:14.870185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b338-4afd-4ab8-84f6-167af669abf0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:14.870748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b338-4afd-4ab8-84f6-167af669abf0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:14.871069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b338-4afd-4ab8-84f6-167af669abf0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:14.872646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:14.879583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847dfed-fbca-4515-a835-3f23f1bed9fa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:14.880070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847dfed-fbca-4515-a835-3f23f1bed9fa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:14.880180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847dfed-fbca-4515-a835-3f23f1bed9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:14.901238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9847dfed-fbca-4515-a835-3f23f1bed9fa None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:14.902503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9847dfed-fbca-4515-a835-3f23f1bed9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:14.910802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:14.951465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b338-4afd-4ab8-84f6-167af669abf0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:14.962917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b338-4afd-4ab8-84f6-167af669abf0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:14.967397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:14.968416 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:14.968752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:14.969153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:14.981652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:14.983120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6513d-6e99-48b6-8598-207ca3570ed2 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:14.993326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 715/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:17.415190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c018-9c1e-4874-8585-8277a3e21d17 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:17.415830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c018-9c1e-4874-8585-8277a3e21d17 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:17.415830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995c018-9c1e-4874-8585-8277a3e21d17 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:17.417246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:17.422864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecd5b-0f50-4377-91aa-658717c83571 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:17.423422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecd5b-0f50-4377-91aa-658717c83571 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:17.423422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ecd5b-0f50-4377-91aa-658717c83571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:17.442148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-729ecd5b-0f50-4377-91aa-658717c83571 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:17.443367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-729ecd5b-0f50-4377-91aa-658717c83571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:17.452975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:17.492143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995c018-9c1e-4874-8585-8277a3e21d17 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:17.504627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995c018-9c1e-4874-8585-8277a3e21d17 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:17.509119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:17.510217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:17.510217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:17.510443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:17.520674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:17.522127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c11b6-72f0-430b-ae10-ecf2fa0a0634 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:17.532035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 733/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:19.741327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45b9e-0baa-4b13-b573-12226fbb7e16 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:19.741929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45b9e-0baa-4b13-b573-12226fbb7e16 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:19.741929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45b9e-0baa-4b13-b573-12226fbb7e16 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:19.743222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:19.749128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c591-6630-496d-a320-3dd10921d765 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:19.749632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c591-6630-496d-a320-3dd10921d765 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:19.749632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c591-6630-496d-a320-3dd10921d765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:19.769663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a733c591-6630-496d-a320-3dd10921d765 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:19.770870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a733c591-6630-496d-a320-3dd10921d765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:19.779949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:19.819347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb45b9e-0baa-4b13-b573-12226fbb7e16 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:19.831301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb45b9e-0baa-4b13-b573-12226fbb7e16 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:19.835911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:19.836615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:19.836902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:19.837180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:19.847718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:19.849216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c5692-b95f-4f48-825e-403ebfd76d97 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:19.859633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 718/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:21.974838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf548-93f9-422a-936d-78fd0768059c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:21.975401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf548-93f9-422a-936d-78fd0768059c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:21.975401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf548-93f9-422a-936d-78fd0768059c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:21.976781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:21.983184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb4f8-cb52-4b8f-ae6f-720ffb3a030c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:21.983682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb4f8-cb52-4b8f-ae6f-720ffb3a030c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:21.983682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeb4f8-cb52-4b8f-ae6f-720ffb3a030c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:22.005895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f2eeb4f8-cb52-4b8f-ae6f-720ffb3a030c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:22.007072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2eeb4f8-cb52-4b8f-ae6f-720ffb3a030c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:22.017787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:22.056377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcf548-93f9-422a-936d-78fd0768059c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:22.068707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcf548-93f9-422a-936d-78fd0768059c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:22.072647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:22.073285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:22.073579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:22.073854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:22.083442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:22.084952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498454-3637-4bce-8fb8-0627c00fb522 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:22.095080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 736/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:24.249150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc02501-d3d4-4aa7-a2a8-83789e49a7f9 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:24.249699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc02501-d3d4-4aa7-a2a8-83789e49a7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:24.249699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc02501-d3d4-4aa7-a2a8-83789e49a7f9 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:24.251030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:24.257410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d9bb-c633-43cf-b5e3-42bd8e0edd43 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:24.257909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d9bb-c633-43cf-b5e3-42bd8e0edd43 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:24.258005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d9bb-c633-43cf-b5e3-42bd8e0edd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:24.277242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2870d9bb-c633-43cf-b5e3-42bd8e0edd43 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:24.278477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2870d9bb-c633-43cf-b5e3-42bd8e0edd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:24.287475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:24.326498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc02501-d3d4-4aa7-a2a8-83789e49a7f9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:24.338528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc02501-d3d4-4aa7-a2a8-83789e49a7f9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:24.342677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:24.343652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:24.344081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:24.344459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:24.358592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:24.360059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7278-6453-4882-9512-b70dcfffccf6 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:24.370542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 721/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:26.467837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1970-afcf-49ed-9371-2bf50a43490c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:26.468438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1970-afcf-49ed-9371-2bf50a43490c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:26.468544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b1970-afcf-49ed-9371-2bf50a43490c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:26.470557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:26.478402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180999c-4d44-4b7a-b340-6ad2d9b52225 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:26.479038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180999c-4d44-4b7a-b340-6ad2d9b52225 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:26.479038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180999c-4d44-4b7a-b340-6ad2d9b52225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:26.499818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3180999c-4d44-4b7a-b340-6ad2d9b52225 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:26.501009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3180999c-4d44-4b7a-b340-6ad2d9b52225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:26.510070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:26.550493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b1970-afcf-49ed-9371-2bf50a43490c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:26.567295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b1970-afcf-49ed-9371-2bf50a43490c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:26.573235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:26.573891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:26.574178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:26.574448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:26.583863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:26.585335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54377a9-17b7-4cb1-b0ac-170ce952298b service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:26.595320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 739/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:28.702457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c921-f054-4592-aa89-9354d701e828 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:28.703004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c921-f054-4592-aa89-9354d701e828 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:28.703004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c921-f054-4592-aa89-9354d701e828 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:28.704442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:28.710176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d182-46ad-4c80-b080-13f3d04256b1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:28.710616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d182-46ad-4c80-b080-13f3d04256b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:28.710712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18d182-46ad-4c80-b080-13f3d04256b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:28.717109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=79 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:20:28.731207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fc18d182-46ad-4c80-b080-13f3d04256b1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:28.732462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc18d182-46ad-4c80-b080-13f3d04256b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:28.741824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:28.780789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c921-f054-4592-aa89-9354d701e828 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:28.786695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=73 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:20:28.793272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c921-f054-4592-aa89-9354d701e828 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:28.797203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:28.797885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:28.798170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:28.798452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:28.810722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:28.812196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099c43c-ff26-4102-a3ec-2a271d201962 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:28.822087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 724/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:30.952491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db133165-1a56-4b23-8676-d0d18ed8eb95 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:30.953055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db133165-1a56-4b23-8676-d0d18ed8eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:30.953055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db133165-1a56-4b23-8676-d0d18ed8eb95 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:30.954468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:30.960074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648bbb-b39c-49e4-86a9-e29453c8f98c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:30.960573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648bbb-b39c-49e4-86a9-e29453c8f98c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:30.960686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99648bbb-b39c-49e4-86a9-e29453c8f98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:30.980483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-99648bbb-b39c-49e4-86a9-e29453c8f98c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:30.981714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-99648bbb-b39c-49e4-86a9-e29453c8f98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:30.990159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:31.028545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db133165-1a56-4b23-8676-d0d18ed8eb95 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:31.040981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db133165-1a56-4b23-8676-d0d18ed8eb95 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:31.045193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:31.045845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:31.046130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:31.046385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:31.056467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:31.057840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3b8a9-f12d-4b0e-8cfc-1b2322a798f8 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:31.068568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 742/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:33.097966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9934-bc45-4f4a-8657-9b45d481bfa5 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:33.098905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9934-bc45-4f4a-8657-9b45d481bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:33.098905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a9934-bc45-4f4a-8657-9b45d481bfa5 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:33.101084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:20:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:33.109532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b579-d58f-4f52-8b45-6670c17f84cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:33.110014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b579-d58f-4f52-8b45-6670c17f84cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:33.110118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b579-d58f-4f52-8b45-6670c17f84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:33.130248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5e93b579-d58f-4f52-8b45-6670c17f84cf None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:33.131395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5e93b579-d58f-4f52-8b45-6670c17f84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:33.145029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:33.186953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a9934-bc45-4f4a-8657-9b45d481bfa5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:33.204437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a9934-bc45-4f4a-8657-9b45d481bfa5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:33.209081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:33.209734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:33.210017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:33.210315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:33.220188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:33.221688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dd7f3-7f97-4933-b948-4016d3f57ddf service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:33.233903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 727/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:43.124547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=11 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:20:43.227174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=14 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:20:53.182683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc52a-147f-4a34-8474-23d5afad7318 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:53.183330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc52a-147f-4a34-8474-23d5afad7318 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:53.183330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dc52a-147f-4a34-8474-23d5afad7318 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:53.184677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 744/1471] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:53.190727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21521e-fee6-4ec2-a1f0-cf82842dac11 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:53.191185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21521e-fee6-4ec2-a1f0-cf82842dac11 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:53.191285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21521e-fee6-4ec2-a1f0-cf82842dac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:53.209646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-da21521e-fee6-4ec2-a1f0-cf82842dac11 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:53.210859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-da21521e-fee6-4ec2-a1f0-cf82842dac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:53.219637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 728/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:53.268463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bbc-531c-4c68-8f9f-939365b78edd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:53.268751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bbc-531c-4c68-8f9f-939365b78edd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:53.269026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c64bbc-531c-4c68-8f9f-939365b78edd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:53.270446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 745/1473] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jun 27 15:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:53.275629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556827b-95e3-49e7-840e-054772ea62b7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:53.275921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556827b-95e3-49e7-840e-054772ea62b7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:53.276293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556827b-95e3-49e7-840e-054772ea62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:53.295718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a556827b-95e3-49e7-840e-054772ea62b7 None None] MFA Rules not processed for user `62ae50de406e46dbb2448a8f04426bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:53.296981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a556827b-95e3-49e7-840e-054772ea62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:53.304536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 729/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jun 27 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:53.312648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64bbc-531c-4c68-8f9f-939365b78edd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:53.325170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c64bbc-531c-4c68-8f9f-939365b78edd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:53.328875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:53.329519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:53.329797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:53.330072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:53.341855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:53.343522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3884d79-65de-4c9e-b856-05b690b40c13 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:53.354084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 746/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:20:55.525420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1c5c-dc40-4d64-b2f3-8e1ae1f48584 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:55.526050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1c5c-dc40-4d64-b2f3-8e1ae1f48584 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:55.526347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d1c5c-dc40-4d64-b2f3-8e1ae1f48584 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:55.527742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 730/1476] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:55.533255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047a0d6-cbb3-4c0a-b9f9-e98aef03b5c3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:55.533712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047a0d6-cbb3-4c0a-b9f9-e98aef03b5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:55.533827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047a0d6-cbb3-4c0a-b9f9-e98aef03b5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:55.552953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4047a0d6-cbb3-4c0a-b9f9-e98aef03b5c3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:55.554183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4047a0d6-cbb3-4c0a-b9f9-e98aef03b5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:55.563032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 747/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:55.587484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1c5c-dc40-4d64-b2f3-8e1ae1f48584 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:55.602492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d1c5c-dc40-4d64-b2f3-8e1ae1f48584 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:55.607819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:55.608505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:55.608771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:55.609042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:55.609761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:55.611575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:20:55.611575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:20:55.611575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:20:55.612727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b027e-d269-418b-be4a-1ec3d679df0d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:55.632964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 731/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jun 27 15:20:55 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:20:57.736531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a292-0f5c-4aae-9b43-715197a10e57 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:57.737109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a292-0f5c-4aae-9b43-715197a10e57 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:57.737109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6a292-0f5c-4aae-9b43-715197a10e57 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:57.738549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 748/1479] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:20:57.744310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8350-c749-4dda-999a-2e948adb47f0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:57.744835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8350-c749-4dda-999a-2e948adb47f0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:57.744939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b8350-c749-4dda-999a-2e948adb47f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:57.764531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9f7b8350-c749-4dda-999a-2e948adb47f0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:20:57.765738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f7b8350-c749-4dda-999a-2e948adb47f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:20:57.774365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 732/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:20:57.818737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6a292-0f5c-4aae-9b43-715197a10e57 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:20:57.835944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6a292-0f5c-4aae-9b43-715197a10e57 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:20:57.842163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:20:57.842812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:20:57.843158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:20:57.843381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:20:57.853181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:20:57.854874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dd1ac-7e20-4cd8-80f3-67a5e1bbbdcf service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:20:57.865458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 749/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:21:00.127011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd014fb-fa36-42a0-b789-ef4ab2e27069 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:00.127566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd014fb-fa36-42a0-b789-ef4ab2e27069 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:00.127836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd014fb-fa36-42a0-b789-ef4ab2e27069 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:00.130084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 733/1482] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:21:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:21:00.136118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ba9be-cdab-4da5-bc91-aa21d4dafb41 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:00.136663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ba9be-cdab-4da5-bc91-aa21d4dafb41 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:00.136779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ba9be-cdab-4da5-bc91-aa21d4dafb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:00.156958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-133ba9be-cdab-4da5-bc91-aa21d4dafb41 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:21:00.158197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-133ba9be-cdab-4da5-bc91-aa21d4dafb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:21:00.166691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 750/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:21:00.204130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd014fb-fa36-42a0-b789-ef4ab2e27069 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:21:00.217307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd014fb-fa36-42a0-b789-ef4ab2e27069 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:21:00.221076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:21:00.221733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:00.222679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:00.222679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:00.234914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:21:00.236420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a3384-4627-43fc-85f6-a690f549bf3e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:21:00.246419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 734/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:21:02.358329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf62a-87c3-43cb-b87d-11b7da783e77 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:02.358938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf62a-87c3-43cb-b87d-11b7da783e77 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:02.358938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbf62a-87c3-43cb-b87d-11b7da783e77 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:02.360269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 751/1485] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:21:02.365693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fc436-56af-48e5-a38f-6233da0fc21b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:02.366158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fc436-56af-48e5-a38f-6233da0fc21b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:02.366263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fc436-56af-48e5-a38f-6233da0fc21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:02.385903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fa7fc436-56af-48e5-a38f-6233da0fc21b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:21:02.387116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fa7fc436-56af-48e5-a38f-6233da0fc21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:21:02.395749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 735/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:21:02.436281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbf62a-87c3-43cb-b87d-11b7da783e77 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:21:02.450196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbf62a-87c3-43cb-b87d-11b7da783e77 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:21:02.454050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:21:02.454702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:02.454987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:02.455258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:02.466427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:21:02.467879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3580e-cf20-4db4-bd50-ba6d7bcbec7a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:21:02.478783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 752/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:21:04.612907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ae89a-2ac2-4c56-897f-ea7d7203c4e0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:04.613537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ae89a-2ac2-4c56-897f-ea7d7203c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:04.613537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ae89a-2ac2-4c56-897f-ea7d7203c4e0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:04.614815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 736/1488] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:21:04.620740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770aa96-60b4-4912-89c6-7cd97944914b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:04.621192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770aa96-60b4-4912-89c6-7cd97944914b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:04.621320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770aa96-60b4-4912-89c6-7cd97944914b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:04.640501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8770aa96-60b4-4912-89c6-7cd97944914b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:21:04.641862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8770aa96-60b4-4912-89c6-7cd97944914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:21:04.651600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 753/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:21:04.692395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ae89a-2ac2-4c56-897f-ea7d7203c4e0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:21:04.704630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ae89a-2ac2-4c56-897f-ea7d7203c4e0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:21:04.708710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:21:04.709369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:04.709655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:04.709927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:04.719624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:21:04.721147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515000-8668-400d-ad89-85fb9762cdcb service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:21:04.731181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 737/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:21:06.982362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e585b-8a86-4c7a-a7e5-ea44c076cbda None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:06.983002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e585b-8a86-4c7a-a7e5-ea44c076cbda None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:06.983002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e585b-8a86-4c7a-a7e5-ea44c076cbda None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:06.984345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 754/1491] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jun 27 15:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:21:06.989703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d313c-dd67-413e-8ca8-0acbd8c981bd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:06.990163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d313c-dd67-413e-8ca8-0acbd8c981bd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:06.990270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d313c-dd67-413e-8ca8-0acbd8c981bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:07.008822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8f4d313c-dd67-413e-8ca8-0acbd8c981bd None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:21:07.010207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8f4d313c-dd67-413e-8ca8-0acbd8c981bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:21:07.018828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 738/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jun 27 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:21:07.058510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e585b-8a86-4c7a-a7e5-ea44c076cbda None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:21:07.070656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e585b-8a86-4c7a-a7e5-ea44c076cbda None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:21:07.074750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:21:07.075406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:21:07.075683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:21:07.075957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:21:07.086754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:21:07.088221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740f5c1-dc80-4de7-a66d-b96581ba695f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:21:07.098440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 755/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:21:17.003268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=33 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:21:17.094055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=33 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:22:33.504922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.505569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.505569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.526233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:22:33.527647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:33.537300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 739/1494] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:22:33.546767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.547325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.547325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.575610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:22:33.577532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:33.590217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 756/1495] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:22:33.596585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.607086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48058e10-e942-4fb8-8ecf-3f1f1e2c8f7c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.613091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.613763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.614045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.614318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.615054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:22:33.615365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.616424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.616424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:22:33.616424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:22:33.617425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.629748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 740/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:22:33.638016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.647584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198a97c-56bd-4c8f-bb5f-c5d70c7dcd16 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.653234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.653900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.654181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.654460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.655150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.657018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.657018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:22:33.657018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:22:33.658255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.699752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 757/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:22:33 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:22:33.706832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.721130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f99f-1047-4ee4-b998-93bdff613c99 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.780205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.780869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.781152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.781399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.782093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.783120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.783120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:22:33.783120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:22:33.784366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.807073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 741/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:22:33 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:22:33.811431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.822361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d858b7-1b61-42f6-83e0-caf718ad4a17 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.827060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.827721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.828003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.828277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.829038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.830059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.830059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:22:33.830059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:22:33.830957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.844229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 758/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:22:33.848630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.862423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4dcf-c751-4d7d-9101-c685c77cc196 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.868233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.868930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.869215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.869499 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.870227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.871254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.871254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:22:33.871254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:22:33.872173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411a0c0-d2fa-4c8e-ba23-970d94d64eae admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.886011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 742/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:22:33.890386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:33.900787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ead503-e645-409c-8fda-9186509d4d49 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:33.905654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:33.906533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.906814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.907114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] PATH_INFO: `/v3/projects/fff958b4ff79492c8bfcf5f539dcbd3e/users/ee7ec9316fb342a794f7294a9d723ea6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.924164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff958b4ff79492c8bfcf5f539dcbd3e, user_id=ee7ec9316fb342a794f7294a9d723ea6, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:33.925232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:33.925232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:22:33.925232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:22:33.927160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49883fe8-8c9b-4b29-867f-f7c584e6d98c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:33.942345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 759/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:22:33 2022] PUT /identity/v3/projects/fff958b4ff79492c8bfcf5f539dcbd3e/users/ee7ec9316fb342a794f7294a9d723ea6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:22:33.954601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186e65-f9ee-448b-9d4b-9ca27126ac32 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:33.954892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186e65-f9ee-448b-9d4b-9ca27126ac32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:33.955172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186e65-f9ee-448b-9d4b-9ca27126ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:33.983668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9a186e65-f9ee-448b-9d4b-9ca27126ac32 None None] MFA Rules not processed for user `ee7ec9316fb342a794f7294a9d723ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:22:33.984939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9a186e65-f9ee-448b-9d4b-9ca27126ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:34.054147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 743/1502] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:22:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:22:34.069871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe37ef-6f0d-4526-8b65-88ae07da37ad None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.070172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe37ef-6f0d-4526-8b65-88ae07da37ad None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.070781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe37ef-6f0d-4526-8b65-88ae07da37ad None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.072177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 760/1503] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jun 27 15:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:22:34.077268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b331-4718-4660-92d8-e51ae9d1e39e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.077559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b331-4718-4660-92d8-e51ae9d1e39e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.077843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0b331-4718-4660-92d8-e51ae9d1e39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.097926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-acf0b331-4718-4660-92d8-e51ae9d1e39e None None] MFA Rules not processed for user `4b9a1a77fd5b40f6b531c79d69044430`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:22:34.099146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-acf0b331-4718-4660-92d8-e51ae9d1e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:34.151554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:22:34.159962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 744/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jun 27 15:22:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:22:34.168255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe37ef-6f0d-4526-8b65-88ae07da37ad None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:34.180894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe37ef-6f0d-4526-8b65-88ae07da37ad None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:34.185414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:34.186057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.186338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.186608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.196351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:34.197831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:34.206815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 761/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:22:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:22:34.257686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e73e-2b40-4e19-92a2-5492b01c2173 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.258059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e73e-2b40-4e19-92a2-5492b01c2173 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.258366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60e73e-2b40-4e19-92a2-5492b01c2173 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.259667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 745/1506] 10.222.0.26 () {52 vars in 829 bytes} [Mon Jun 27 15:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:22:34.263446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:34.334150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c69a-4465-43d6-9011-66ee3f8e5fb1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:34.340097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:34.341017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.341289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.341550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] PATH_INFO: `/v3/users/ee7ec9316fb342a794f7294a9d723ea6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.342826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee7ec9316fb342a794f7294a9d723ea6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:34.343841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:34.343841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:22:34.343841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:22:34.345095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:34.425778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee7ec9316fb342a794f7294a9d723ea6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:34.426996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee7ec9316fb342a794f7294a9d723ea6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:34.428010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee7ec9316fb342a794f7294a9d723ea6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:34.442425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee7ec9316fb342a794f7294a9d723ea6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:34.454735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee7ec9316fb342a794f7294a9d723ea6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:34.471378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:22:34.484523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 762/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:22:34 2022] DELETE /identity/v3/users/ee7ec9316fb342a794f7294a9d723ea6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:22:34.486311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.486611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.486892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.507443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:22:34.509262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:34.578494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 746/1508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:22:34.589093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:34.590648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:34.655655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce5ef-91a0-40a0-9828-ecf1df20698f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:34.661432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:34.662416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:34.662796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:34.663159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:34.673886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:34.675347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba32f3-6b70-4794-8c28-f92a12aa4fc0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:34.685252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 763/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:22:35.036510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:22:35.038732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:22:35.054456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb679912-ed86-46b9-8479-29aa36af380e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:22:35.060043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:22:35.061197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:22:35.061550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:22:35.061895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] PATH_INFO: `/v3/projects/fff958b4ff79492c8bfcf5f539dcbd3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:22:35.063630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff958b4ff79492c8bfcf5f539dcbd3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:22:35.064801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:22:35.064801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:22:35.064801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:22:35.066984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:22:35.131599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff958b4ff79492c8bfcf5f539dcbd3e', 'request_id': 'req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:35.145555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff958b4ff79492c8bfcf5f539dcbd3e', 'request_id': 'req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:35.157444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff958b4ff79492c8bfcf5f539dcbd3e', 'request_id': 'req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:35.170637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] The token cache is being invalidate because project fff958b4ff79492c8bfcf5f539dcbd3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:22:35.170929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:35.171567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3851bbd1-34c9-407b-b7c5-7181dfdee9d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:22:35.173285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 747/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:22:35 2022] DELETE /identity/v3/projects/fff958b4ff79492c8bfcf5f539dcbd3e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:22:44.680712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:22:45.166327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:24:12.673752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:12.674401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:12.674401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:12.675796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:12.681026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71c92-d05b-446f-b4a1-ae5d39c10979 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:12.681839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71c92-d05b-446f-b4a1-ae5d39c10979 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:12.681839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b71c92-d05b-446f-b4a1-ae5d39c10979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:12.701019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-02b71c92-d05b-446f-b4a1-ae5d39c10979 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:12.702245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-02b71c92-d05b-446f-b4a1-ae5d39c10979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:12.777603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:12.937646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:12.939745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:13.019753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec37ce-9157-4b79-86a7-fd0d5ce3a69f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:13.025769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:13.026916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:13.027195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:13.027472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:13.036269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:13.037920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442a8bb-5053-47f9-9e42-149f85fe31d8 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:13.053137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 765/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:13.912849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bf949-7677-4406-a4e2-24991de583c6 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:13.913733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bf949-7677-4406-a4e2-24991de583c6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:13.913733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bf949-7677-4406-a4e2-24991de583c6 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:13.915464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:13.921025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830e11a-0dbc-49b4-adea-7da295a8500d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:13.921583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830e11a-0dbc-49b4-adea-7da295a8500d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:13.921583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830e11a-0dbc-49b4-adea-7da295a8500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:13.942079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8830e11a-0dbc-49b4-adea-7da295a8500d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:13.943330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8830e11a-0dbc-49b4-adea-7da295a8500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:13.952646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:14.111069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bf949-7677-4406-a4e2-24991de583c6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:14.130687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bf949-7677-4406-a4e2-24991de583c6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:14.137171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:14.137937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:14.138220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:14.138494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:14.147568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:14.149222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba33c1-51a6-4872-a019-bf8a5b214bca service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:14.159645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 750/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:14.837826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1056-82b6-4880-9dbd-0dfdf15bee72 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:14.838185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1056-82b6-4880-9dbd-0dfdf15bee72 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:14.838534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d1056-82b6-4880-9dbd-0dfdf15bee72 None None] PATH_INFO: `/v3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:14.840298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 767/1517] 10.222.0.42 () {54 vars in 925 bytes} [Mon Jun 27 15:24:14 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:24:14.847865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac76f9-5003-4603-942f-a0b47faeeead None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:14.848153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac76f9-5003-4603-942f-a0b47faeeead None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:14.848480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac76f9-5003-4603-942f-a0b47faeeead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:14.868283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fdac76f9-5003-4603-942f-a0b47faeeead None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:14.869572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fdac76f9-5003-4603-942f-a0b47faeeead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:14.950498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 751/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jun 27 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:18.739672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bf9b-7735-4ed1-860b-3e1b990291b8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:18.740566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bf9b-7735-4ed1-860b-3e1b990291b8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:18.740566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bf9b-7735-4ed1-860b-3e1b990291b8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:18.742580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 768/1519] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jun 27 15:24:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:18.750993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3854b4-91a1-407c-9c70-deadd3febd28 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:18.751736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3854b4-91a1-407c-9c70-deadd3febd28 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:18.751904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3854b4-91a1-407c-9c70-deadd3febd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:18.775823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ce3854b4-91a1-407c-9c70-deadd3febd28 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:18.777053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ce3854b4-91a1-407c-9c70-deadd3febd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:18.788050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 752/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jun 27 15:24:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:24:18.796193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6bf9b-7735-4ed1-860b-3e1b990291b8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:18.820063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6bf9b-7735-4ed1-860b-3e1b990291b8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:18.828481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:18.829137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:18.829421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:18.829679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:18.838928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:18.840581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fb961-b810-4c01-bb0b-e9d5c70c3694 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:18.852588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 769/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:25.438266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8deb-43b9-454d-b1ef-9c19c0ef9a96 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:25.438873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8deb-43b9-454d-b1ef-9c19c0ef9a96 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:25.438873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc8deb-43b9-454d-b1ef-9c19c0ef9a96 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:25.440179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:25.446152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbc5b-33a0-45aa-bf14-100a57b27486 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:25.447138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbc5b-33a0-45aa-bf14-100a57b27486 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:25.447138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbc5b-33a0-45aa-bf14-100a57b27486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:25.469177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3e9dbc5b-33a0-45aa-bf14-100a57b27486 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:25.470388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e9dbc5b-33a0-45aa-bf14-100a57b27486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:25.480681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:25.642118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8deb-43b9-454d-b1ef-9c19c0ef9a96 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:25.659596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc8deb-43b9-454d-b1ef-9c19c0ef9a96 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:25.665909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:25.666561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:25.666848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:25.667124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:25.675934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:25.677567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a599cf3-ce7c-4c3e-ae34-1afc72edef3e service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:25.687844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 754/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:26.505812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae7210-1049-4b24-a10d-dc2412d32e59 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:26.506700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae7210-1049-4b24-a10d-dc2412d32e59 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:26.506700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae7210-1049-4b24-a10d-dc2412d32e59 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:26.509043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:24:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:26.517465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe4f1a-a267-47e1-ac22-7093d23c0016 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:26.518222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe4f1a-a267-47e1-ac22-7093d23c0016 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:26.518418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe4f1a-a267-47e1-ac22-7093d23c0016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:26.542533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-92fe4f1a-a267-47e1-ac22-7093d23c0016 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:26.543749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-92fe4f1a-a267-47e1-ac22-7093d23c0016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:26.552796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:26.706761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae7210-1049-4b24-a10d-dc2412d32e59 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:26.722733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae7210-1049-4b24-a10d-dc2412d32e59 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:26.728824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:26.729462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:26.729757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:26.730082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:26.738685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:26.740307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634e469-c95b-48e6-b4d3-1a2c7dfd1a5e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:26.749708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 772/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:30.769747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:30.770362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:30.770362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:30.771712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:30.777427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58eb1-3df1-45a3-a65d-88d38ae40172 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:30.777873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58eb1-3df1-45a3-a65d-88d38ae40172 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:30.777968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec58eb1-3df1-45a3-a65d-88d38ae40172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:30.797814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5ec58eb1-3df1-45a3-a65d-88d38ae40172 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:30.798935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ec58eb1-3df1-45a3-a65d-88d38ae40172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:30.807331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:30.845971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:30.848279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:30.916954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dada6-952f-4167-ac89-8dab69fda0bd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:30.921726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:30.922393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:30.922659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:30.922955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:30.933718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:30.935180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158164f-7b4a-4a9e-b7e4-5f14346ad43c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:30.941639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:24:30.948919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 757/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:32.981408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3036-4135-4349-8375-09bf4a06993a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:32.982412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3036-4135-4349-8375-09bf4a06993a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:32.982412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3036-4135-4349-8375-09bf4a06993a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:32.984526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:24:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:32.991176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c858c20-d347-4054-b432-2121a7bc692a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:32.991956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c858c20-d347-4054-b432-2121a7bc692a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:32.992114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c858c20-d347-4054-b432-2121a7bc692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:33.018835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6c858c20-d347-4054-b432-2121a7bc692a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:33.020093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6c858c20-d347-4054-b432-2121a7bc692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:33.029659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:33.077128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3036-4135-4349-8375-09bf4a06993a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:33.090830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3036-4135-4349-8375-09bf4a06993a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:33.095821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:33.096512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:33.096795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:33.097070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:33.107299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:33.108854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d653d9-01b2-415a-8786-6886b9b4cf53 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:33.118868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 775/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:36.650826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:36.651393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:36.651393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:36.652806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:36.658999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750209-ef1f-407e-bc3e-215b365948d4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:36.659522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750209-ef1f-407e-bc3e-215b365948d4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:36.659632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5750209-ef1f-407e-bc3e-215b365948d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:36.670602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=62 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:24:36.686122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b5750209-ef1f-407e-bc3e-215b365948d4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:36.687651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5750209-ef1f-407e-bc3e-215b365948d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:36.696604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:24:37.005880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:37.008281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:37.079486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c982c-ab82-478d-b63a-7ebb33afcaaa None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:37.083390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:37.084053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:37.084344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:37.084642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:37.094507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:37.095955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee91e56-a82a-4c03-8794-d6e2b6c9c181 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:37.106320 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 760/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:39.190726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:39.191315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:39.191315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:39.193134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:24:39.201485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b58e1-d7f2-439a-9f8a-82ac75936bd2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:39.202052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b58e1-d7f2-439a-9f8a-82ac75936bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:39.202835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b58e1-d7f2-439a-9f8a-82ac75936bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:39.222889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ef3b58e1-d7f2-439a-9f8a-82ac75936bd2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:24:39.224170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ef3b58e1-d7f2-439a-9f8a-82ac75936bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:39.233933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:24:39.549857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:24:39.552195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:24:39.575342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2fe4d-b3c7-4419-8c88-4dff9075e753 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:24:39.580545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:24:39.581201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:24:39.581478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:24:39.581750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:24:39.595582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:24:39.597065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e057c1a-b8bb-43e2-a471-7026c85a311c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:24:39.610667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 778/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:24:49.216759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=32 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:24:49.604386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:11.462036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db9f62-2532-418c-9874-aee074172281 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.464141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db9f62-2532-418c-9874-aee074172281 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.464970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db9f62-2532-418c-9874-aee074172281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.466934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a10ac-fbb9-429b-9fe4-bc435c8d4795 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.467841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a10ac-fbb9-429b-9fe4-bc435c8d4795 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.467841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a10ac-fbb9-429b-9fe4-bc435c8d4795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.507537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7d2a10ac-fbb9-429b-9fe4-bc435c8d4795 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.509530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7d2a10ac-fbb9-429b-9fe4-bc435c8d4795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.511772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d1db9f62-2532-418c-9874-aee074172281 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.514368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d1db9f62-2532-418c-9874-aee074172281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.535995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 779/1540] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:26:11.539519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 762/1541] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.547670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157004e5-a7bc-476f-86d6-a6f78e9154ed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.548150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157004e5-a7bc-476f-86d6-a6f78e9154ed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.548642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157004e5-a7bc-476f-86d6-a6f78e9154ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.579760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.580916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.581915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.595064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-157004e5-a7bc-476f-86d6-a6f78e9154ed None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.604380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-157004e5-a7bc-476f-86d6-a6f78e9154ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.637853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 780/1542] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.642749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.644525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054c4c-b68e-43fb-b353-51fd62735b3f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.645560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054c4c-b68e-43fb-b353-51fd62735b3f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.647046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.648260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054c4c-b68e-43fb-b353-51fd62735b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.691057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 763/1543] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.693203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:11.728522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1c054c4c-b68e-43fb-b353-51fd62735b3f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.734599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1c054c4c-b68e-43fb-b353-51fd62735b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.741957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b689e9-cace-4947-a09f-1f685976959b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:11.768345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:11.771800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.774776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 781/1544] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.775566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.776206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.782227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:11.785605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:11.788375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4df2fa-beaa-4ba2-8afc-7dcd0fbaaa2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.789060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4df2fa-beaa-4ba2-8afc-7dcd0fbaaa2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.789666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4df2fa-beaa-4ba2-8afc-7dcd0fbaaa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.793934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:11.857489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 764/1545] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:11.860928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:11.879603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dc4df2fa-beaa-4ba2-8afc-7dcd0fbaaa2f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.885413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dc4df2fa-beaa-4ba2-8afc-7dcd0fbaaa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.909149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0cc1-56ed-4f16-9b04-3ce471e6ec75 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:11.912062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 782/1546] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.914485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.914972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.915435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.927140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:11.928282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.928781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.929264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.931013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:11.931543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:11.934754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716aa28-5103-4ef2-965d-44cc382adda0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:11.946694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.948680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.962516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 765/1547] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:11.962516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056ba3-a97b-4515-8ff4-a9242c6fb274 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.962516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056ba3-a97b-4515-8ff4-a9242c6fb274 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.962516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77056ba3-a97b-4515-8ff4-a9242c6fb274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.963001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 783/1548] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:11.963939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:11.986020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d890a62-20cc-4d77-b85e-9468621c1a5f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:11.992358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-77056ba3-a97b-4515-8ff4-a9242c6fb274 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:11.993562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:11.994523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77056ba3-a97b-4515-8ff4-a9242c6fb274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:11.994666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:11.995080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:11.995528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:11.996805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.000523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e58f3-3972-4c91-92f3-741aa48977b1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.009257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 766/1549] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.011270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a312a-5960-458d-86c0-436d673fb6cb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.011725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a312a-5960-458d-86c0-436d673fb6cb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.012174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a312a-5960-458d-86c0-436d673fb6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.044295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f25a312a-5960-458d-86c0-436d673fb6cb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.046205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f25a312a-5960-458d-86c0-436d673fb6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.051870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 784/1550] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:11 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.051870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.051870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.051870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.079185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.081925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.162698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 767/1551] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.164698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.165066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.165351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.183278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 785/1552] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.184146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.193448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.194734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.195755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3132897-a83d-4fc3-8448-9bd10eaedc0e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.202100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.203169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.203628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.204076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.205302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.216395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 768/1553] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.216688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf41d9e-3f6d-4bf1-aca2-a7f8cb3b7469 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.216793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.224004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725574f-a800-4907-81c5-25c68dd48ce1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.229941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.230643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.231000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.231270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.232028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:12.232354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.234043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.239356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 786/1554] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.240486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a191f-8db8-4ac6-bb09-e41101aed62d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.240782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a191f-8db8-4ac6-bb09-e41101aed62d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.241054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a191f-8db8-4ac6-bb09-e41101aed62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.252166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 769/1555] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:12.253219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.261792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8f0a191f-8db8-4ac6-bb09-e41101aed62d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.263047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8f0a191f-8db8-4ac6-bb09-e41101aed62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.272671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a46ed-52d7-46e7-ad92-d9da1156a858 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.346607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 787/1556] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.346607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.346607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.346607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.352217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.353335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.353990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.354426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.355589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.358849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01ed7f-4b0c-48cb-832b-1fae92647a1c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.374340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.375751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.386882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 770/1557] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.387314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 788/1558] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.388215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.388414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca118-d5af-4b52-95e1-d5708bbbb395 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.388618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca118-d5af-4b52-95e1-d5708bbbb395 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.389083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ca118-d5af-4b52-95e1-d5708bbbb395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.405315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607d72d-9ddb-419a-8dea-2342bfa056b9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.410648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-095ca118-d5af-4b52-95e1-d5708bbbb395 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.411715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.411933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-095ca118-d5af-4b52-95e1-d5708bbbb395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.412469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.412757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.413024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.413822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:12.414127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.415286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:12.415286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:26:12.415286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:26:12.416297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.421286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 771/1559] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.426667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.426667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.426667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.435642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 789/1560] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:12.437640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.444216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.445689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.448793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:12.455195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05aaac-0c57-4453-b24c-c7983deeca36 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.456120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 772/1561] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.460145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.466915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.467715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.468442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.468546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.469228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.470314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:12.470314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:26:12.470314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:26:12.471778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.475772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab680a5-0c56-4bfb-b6af-a47c63954cbf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.481553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.482564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.483197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.483197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.483824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.485074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:12.485074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:26:12.485074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:26:12.486323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.496122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:12.503857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 790/1562] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/users => generated 293 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.503857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.516110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd949e-bb88-4b36-9e00-c97eafb888db admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.521379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.522085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.522334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.522611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.523357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:12.523664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.527177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 773/1563] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.527177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.527494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cb924-42cf-4d14-af89-2580180dfadf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.540811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 791/1564] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:12.542629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.542743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.542991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.566733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:12.568008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:12.610270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e8dbc-74e5-4b24-8587-14636707009a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.620440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.621421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.621777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.622108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.623112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.624917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.637516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 774/1565] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:12.638523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.656223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa402728-c686-433c-8a72-0da5f8952a4d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.659942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 792/1566] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:12.662741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.663851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.663996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.664160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.664587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.665167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:12.665484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.667179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.682392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 775/1567] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:12.683453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d5d0f-e176-4b72-8a7c-2f57edbdbd5b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.683651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.692883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.694248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.694751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.695255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.696657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.697728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21f505-303d-4a7a-a5c5-ed003c9961a4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.700310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.703151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.703850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.704134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.704452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.705220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:12.705529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.707222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.721887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 776/1568] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:12.723804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.739875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a776c2-d844-4633-864e-949ac050336e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.743103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 793/1569] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.744013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.749938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.750910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.751368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.751805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.753217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.755969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.778841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 777/1570] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:12.778841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.794206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be732e-8d75-49e6-b657-53187ac2011c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.841334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd9fec-23d8-4dc8-a852-1a954c0480c5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.849071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.850059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.850428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.850900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.851871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.854458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.873587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.874280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.874557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.874834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.875647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.880228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.887871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 794/1571] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.888868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.901248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8ad9a-657c-4ff3-9322-b305c7d52232 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.907051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:12.907872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:12.908167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:12.908547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:12.910005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:12.911746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:12.927317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 795/1572] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:12.928541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.938333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 778/1573] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:12.938333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:12.944003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b41074-fd8d-44b6-9030-e817aec04b59 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:12.985867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:12.989715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:13.030800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.031691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.032151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.032678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.033173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8d885-e0e1-4346-a4b2-07c2ad20a104 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.033733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:13.034228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.036461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.038894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.039630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.040069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.040577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.041421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.043748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.053106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 796/1574] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:13.054236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.068237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc535ca-8ea7-42ff-b8b3-f220e7384b62 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.075098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.076142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.076656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.077012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.077932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.080515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.090594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 779/1575] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:12 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.090594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.124783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 797/1576] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.124783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.186718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24256e-fa08-4bf0-923e-54292dc5c1f6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.216931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b45af-2436-40ec-bfd6-0975a7a7c400 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.223908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.225057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.225464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.225818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.226687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.228611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.251191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 798/1577] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.252559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.268947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3ec02-5abe-4d1d-931a-ecbe8f0b3450 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.272767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.273591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.273975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.274348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.275200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.276536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.277243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.277580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.277810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.278572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.280247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.280572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.302557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 799/1578] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.302557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.306013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 780/1579] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.307172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.320868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9fd33-3261-4c96-bd1e-3242c391fe84 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.321222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530321c-fdbc-490b-a2d1-5a29cd6d7153 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.331509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.332487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.332839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.333294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.334257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.334898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.336536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.336760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.336990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.337561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] PATH_INFO: `/v3/projects/090e27bd88cc4d91a6e06fc8a93d56eb/users/fecf8c076b2e4553aba77b7eef4296a7/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.360281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 781/1580] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.364509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.370874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=090e27bd88cc4d91a6e06fc8a93d56eb, user_id=fecf8c076b2e4553aba77b7eef4296a7, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.375799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd06231-cd21-4179-8a82-e585ea09f2c0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.377026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.381899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.382702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.382959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.383281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.384020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.386378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.403089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 800/1581] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:13 2022] PUT /identity/v3/projects/090e27bd88cc4d91a6e06fc8a93d56eb/users/fecf8c076b2e4553aba77b7eef4296a7/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:13.405038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.444001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 782/1582] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/projects => generated 389 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.445428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.475877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:13.491027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca82e39-39d9-4f57-af25-a3c687a3a06d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.504124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:13.536299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9803861-0c37-474c-93ea-1d46cb726cfc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.541274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.541985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.542263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.542539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.543245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.545300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.573655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 783/1583] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.573655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.583004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28ad0-5cd7-4795-9ae4-ddc5e7a6e548 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.594471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.595398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.595682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.595959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] PATH_INFO: `/v3/projects/b97d6836a163455b8e1b87eef3cbebe1/users/add81c0b6df34fb3a93d57b6c258e3e9/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.605572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.606280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.606566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.606841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.607560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.609644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.631681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97d6836a163455b8e1b87eef3cbebe1, user_id=add81c0b6df34fb3a93d57b6c258e3e9, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.633707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:13.633707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:26:13.633707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:26:13.634397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 801/1584] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.635263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.638128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.649271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e4406-22cd-4b15-b37f-bcd7833618c0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.663250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 784/1585] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:13 2022] PUT /identity/v3/projects/b97d6836a163455b8e1b87eef3cbebe1/users/add81c0b6df34fb3a93d57b6c258e3e9/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:13.664768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.748951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.750052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.750446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.750881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.752144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.754675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.759311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f190e5-b630-4b8e-800f-f44a616feb9d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.767150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.768558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.769012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.769485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.770735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.772015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 802/1586] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.773477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.773769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202f77-bffc-4dcf-857d-9de4fe2b55a8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.791324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11e1c0-9b96-496f-acf8-402b2a83d3df admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.791988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 785/1587] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.793615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de904d0-c7ae-444e-917d-19bb6b8d8af6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.794028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de904d0-c7ae-444e-917d-19bb6b8d8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.794401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de904d0-c7ae-444e-917d-19bb6b8d8af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.802713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.803754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.804200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.804764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.805545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.808467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.828287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 803/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.830051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.840157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3de904d0-c7ae-444e-917d-19bb6b8d8af6 None None] MFA Rules not processed for user `fecf8c076b2e4553aba77b7eef4296a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:13.842190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3de904d0-c7ae-444e-917d-19bb6b8d8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:13.851558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c8e9-2a55-4876-a642-f89f84a41ffa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.859193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.860597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.861013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.861459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.863019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.866070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.891355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 804/1589] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/users => generated 295 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:13.892586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.906926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a836c-e00c-4700-85c5-aa4d6946a98a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.914181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.915291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.915736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.916177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.917854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.919904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.932349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 805/1590] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.934484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:13.950174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcb9d-62da-454d-bcc2-136e037a3363 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:13.956113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 786/1591] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:13.958250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:13.959366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.959820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.960049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:13.960390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.960998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:13.961694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:13.961902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:13.964745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:13.982571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:13.983514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 806/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:13.985097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.000309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f65055-788f-44ce-a486-71c3927b26fb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.007551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.009009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.009457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.009897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] PATH_INFO: `/v3/projects/05e46da777a54cbe92cbfb00c37d8bb5/users/5e2597ff867544f9953c519bf29a9bfa/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.012783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] MFA Rules not processed for user `add81c0b6df34fb3a93d57b6c258e3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:14.014721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:14.034482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e46da777a54cbe92cbfb00c37d8bb5, user_id=5e2597ff867544f9953c519bf29a9bfa, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.038839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.058542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 807/1593] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:13 2022] PUT /identity/v3/projects/05e46da777a54cbe92cbfb00c37d8bb5/users/5e2597ff867544f9953c519bf29a9bfa/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:14.060493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.099842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:14.120584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 787/1594] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:14.122058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.137803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6262b-8db1-4611-95ce-882541e7c1dc None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.152291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e34b9-fcaf-456a-bdd1-0b8edc4f4a4d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.157949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.158724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.159194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.159658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.160988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.163025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.176575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 808/1595] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.178276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.192839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb0e65-1493-46a9-a296-ab43baa34f74 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.198706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.199512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.199961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.200403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.201648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.203708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.216534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.217106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 809/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.222239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.222239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.222239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.222842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.223005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.223005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.234584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c2d9-e74c-4214-8a73-b383351232d4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.239853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.240587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.240905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.241145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.241921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.242999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 788/1597] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.243555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.244262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.259678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 810/1598] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.260617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.272048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a7c44-4ec3-428f-8ed5-03576e0a5f93 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.279194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.280163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.280551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.280901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.281798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.283886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80694-9a24-4ca7-aaa2-75315792cf27 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.297723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 811/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.299079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046f74-e800-4390-8f27-529122b02559 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.300049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046f74-e800-4390-8f27-529122b02559 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.300563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea046f74-e800-4390-8f27-529122b02559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.324541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ea046f74-e800-4390-8f27-529122b02559 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:14.326137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea046f74-e800-4390-8f27-529122b02559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:14.328992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a0fd8-be6a-4d1c-8441-d621787ce243 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.333449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.334133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.334418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.334694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.336679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 812/1600] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:14.337884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea046f74-e800-4390-8f27-529122b02559 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.354370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea046f74-e800-4390-8f27-529122b02559 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.362314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.363993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.364561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.365038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.366341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.370517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.411439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:14.416088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 813/1601] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:14 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:14.417428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.430553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.432051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.443466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 789/1602] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:14.444486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.467798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28ee7-6904-4a70-a289-0e5f1f5397ca service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.511596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21880b14-1972-494c-b2f1-e60104fc5bd8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.517290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.519047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.519047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.519656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] PATH_INFO: `/v3/projects/074c7de8debf489aa5c073a1ae623259/users/b952197a0c3e417eb6fe206f2954743f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.539665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=074c7de8debf489aa5c073a1ae623259, user_id=b952197a0c3e417eb6fe206f2954743f, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.543785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.558936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.560476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.560902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.561358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.562519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.564690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.572909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 814/1603] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:14 2022] PUT /identity/v3/projects/074c7de8debf489aa5c073a1ae623259/users/b952197a0c3e417eb6fe206f2954743f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:14.573952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.591030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 790/1604] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:14.591030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.619331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444aaea3-69ca-4ffd-8e8e-f646df0dd6a8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.680574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:14.710670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdc0d1-cca8-406a-a6cf-9b0707820c1d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.716899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.718110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.718615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.719122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] PATH_INFO: `/v3/projects/2e07fc46cd5f4afe875767c926bd63ee/users/995fa1760f5241588a493e5f894b0e3f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.728356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.729606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.730109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.730547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] PATH_INFO: `/v3/projects/2832173587a6494eb7f73b3ca0a09a0f/users/effe0200dc584f11923e83ed240138ff/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.741538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e07fc46cd5f4afe875767c926bd63ee, user_id=995fa1760f5241588a493e5f894b0e3f, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.745286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.754271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2832173587a6494eb7f73b3ca0a09a0f, user_id=effe0200dc584f11923e83ed240138ff, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:14.758599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:14.766251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 815/1605] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:14 2022] PUT /identity/v3/projects/2e07fc46cd5f4afe875767c926bd63ee/users/995fa1760f5241588a493e5f894b0e3f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:14.767769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.786146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 791/1606] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:14 2022] PUT /identity/v3/projects/2832173587a6494eb7f73b3ca0a09a0f/users/effe0200dc584f11923e83ed240138ff/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:14.790400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:14.891418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac55ff6-2eee-4b21-a69d-6f3ff0d165b7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.891691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101e5c6-c258-474a-8bf7-78805606298c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:14.897406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.898382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.898837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:14.899298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:14.943089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:14.996679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:14.998723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:14.999398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.000796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] PATH_INFO: `/v3/projects/e9c75d75a0ea45818f28071163cb93ad/users/d32af5ae84b34f079b84bc416fb5b576/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.040693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c75d75a0ea45818f28071163cb93ad, user_id=d32af5ae84b34f079b84bc416fb5b576, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.046072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.048535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.051583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8bd8e-7e44-45c1-b527-9477f2e70e92 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.080452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 816/1607] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:14 2022] PUT /identity/v3/projects/e9c75d75a0ea45818f28071163cb93ad/users/d32af5ae84b34f079b84bc416fb5b576/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:15.083850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.084559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 792/1608] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:15.086591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.086891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.087210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.162981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] MFA Rules not processed for user `b952197a0c3e417eb6fe206f2954743f`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.165450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.263888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:15.304883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5bbbb-c77e-4b20-9cd1-db2860a4e3b8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.330113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.333274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.333883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.337796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.339116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.344957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80907c6e-b99c-46cc-b9fa-6bf96b0671bc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.356576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 793/1609] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:15.358674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.399866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 817/1610] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/users => generated 300 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:15.401790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe682-6e80-487a-815b-983dd97e6ee5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.402081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe682-6e80-487a-815b-983dd97e6ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.402353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7abe682-6e80-487a-815b-983dd97e6ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.430689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a7abe682-6e80-487a-815b-983dd97e6ee5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.432100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a7abe682-6e80-487a-815b-983dd97e6ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.438376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6e934-4c37-4c7f-ba38-4e69c285c7c3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.445128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 818/1611] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:15.448464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.449370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.450720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.456267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.459070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.461186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.461592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] PATH_INFO: `/v3/projects/2270daf238a640609e9ab514b01d9768/users/e9dab30db0a44655828384db53df29f9/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.488952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2270daf238a640609e9ab514b01d9768, user_id=e9dab30db0a44655828384db53df29f9, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.492474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f759c03-4b89-491c-a220-2d9cd68271f5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.513596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.515801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.517855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 794/1612] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:15 2022] PUT /identity/v3/projects/2270daf238a640609e9ab514b01d9768/users/e9dab30db0a44655828384db53df29f9/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:15.522449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72198da5-4c2b-4f94-a61e-537ebd7c4336 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.524584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72198da5-4c2b-4f94-a61e-537ebd7c4336 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.526394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72198da5-4c2b-4f94-a61e-537ebd7c4336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.571390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-72198da5-4c2b-4f94-a61e-537ebd7c4336 None None] MFA Rules not processed for user `d32af5ae84b34f079b84bc416fb5b576`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.573472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-72198da5-4c2b-4f94-a61e-537ebd7c4336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.650284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 819/1613] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:15.651814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.673239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da8f75-6abb-4746-b5f9-944a9942bb47 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.682217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.683575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.684563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.685493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.687563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.691666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d0da-cf10-486b-9c84-8e5a5f1c6075 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.698849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 795/1614] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:15.700572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.700854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.701138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.702918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 796/1615] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jun 27 15:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:26:15.704673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.708462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.726426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 820/1616] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:15.728132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.730474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.730681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:15.732183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.789775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.792913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.803505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd74686-3cc8-4fe4-b067-7813ad8f5e7d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.806946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 821/1617] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:15.808250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.808692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.809771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.810517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.810864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.821332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d38c1-8dd9-4084-9799-64812054dc7f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.825775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.827697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.828550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9e839-baff-490f-8247-7abfc9adc869 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.829020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.830227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.830901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.844746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.846722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.853413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 797/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:15.856718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.857704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.858427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.859182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 822/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:15.860149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:15.871258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce806-ab83-4247-b0dd-dddc49f442bc service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:15.879303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:15.880442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.880826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.881184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] PATH_INFO: `/v3/projects/861118cee4ba49878a27ab313b4f1cba/users/c75421fe678a4895a4e8bc1adb4e42e5/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.902732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=861118cee4ba49878a27ab313b4f1cba, user_id=c75421fe678a4895a4e8bc1adb4e42e5, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:15.904938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:15.907259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa238b5b-c3ae-437c-965b-39b2ef496ceb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:15.908120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:15.939002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 823/1620] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:15 2022] PUT /identity/v3/projects/861118cee4ba49878a27ab313b4f1cba/users/c75421fe678a4895a4e8bc1adb4e42e5/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:15.941316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce94fd-fe71-4357-82f2-c2fbc550d61e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:15.941316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce94fd-fe71-4357-82f2-c2fbc550d61e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:15.942048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce94fd-fe71-4357-82f2-c2fbc550d61e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:15.951547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 824/1621] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jun 27 15:26:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:26:15.952680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce94fd-fe71-4357-82f2-c2fbc550d61e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:16.042895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce94fd-fe71-4357-82f2-c2fbc550d61e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:16.048027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:16.049021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.050409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.050409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.151928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 798/1622] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jun 27 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:16.153401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:16.185699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:16.188873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593cd19-7658-4dc7-829c-b282fc541bbd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:16.294037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:16.302019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917fa15-a001-43e0-bc96-67363431fe70 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:16.320218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:16.321231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.321517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.321822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.322651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:16.325604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:16.334990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 825/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:16.336276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.336591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.337537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.343760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 799/1624] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:16.345227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:16.387934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:16.389285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:16.482095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:16.575585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 826/1625] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jun 27 15:26:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:16.576454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:16.595933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34f243-da9b-445c-96fc-1f5aab866a4d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:16.605369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85ecd-bb8e-4082-9248-bb98919099ec admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:16.615784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:16.617000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.617494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.617832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] PATH_INFO: `/v3/projects/861118cee4ba49878a27ab313b4f1cba/users/c75421fe678a4895a4e8bc1adb4e42e5/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.620540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861118cee4ba49878a27ab313b4f1cba, user_id=c75421fe678a4895a4e8bc1adb4e42e5, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:16.624759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:16.625454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:16.626601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.627202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.627822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.643796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 827/1626] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:16 2022] PUT /identity/v3/projects/861118cee4ba49878a27ab313b4f1cba/users/c75421fe678a4895a4e8bc1adb4e42e5/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:16.645327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:16.808370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:16.816012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8513-0eb3-4610-8e9e-4df8af8227f5 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:16.883554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 800/1627] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:16.886229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.886700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.887164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.922690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65745abf-708f-47ca-bc0a-e5f65f204450 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:16.955392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:16.956741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] MFA Rules not processed for user `c75421fe678a4895a4e8bc1adb4e42e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:16.958262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:16.959264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:16.959398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:16.959828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:17.098104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 801/1628] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:17.113745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:17.149662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:17.175173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:17.176957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:17.204972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 828/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:17.226679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedf57e-dc9c-4820-8ec9-5295a68a94c2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:17.235018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:17.236421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.237327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.237931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.255601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:17.258484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9b286-7fe8-47ba-b048-bbd17134f15d service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:17.276932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 802/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:17.373100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:17.544957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179df1a-d275-49fd-9bf9-3a7e30229665 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.545447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179df1a-d275-49fd-9bf9-3a7e30229665 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.546026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179df1a-d275-49fd-9bf9-3a7e30229665 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.547736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 803/1631] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jun 27 15:26:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:26:17.556879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.557348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.557729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.564021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aca4a-4b2e-46b5-8818-d5091c9c9f96 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:17.578547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:17.581339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.581339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.582231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] PATH_INFO: `/v3/users/add81c0b6df34fb3a93d57b6c258e3e9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.587261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] RBAC: Authorizing `identity:delete_user(user_id=add81c0b6df34fb3a93d57b6c258e3e9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:17.589986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:17.591196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:17.607067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:17.610237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:17.642412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 804/1632] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:26:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:17.660114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:17.680224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ea9d5-f909-4c0a-8676-f15d00d5f724 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:17.682029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add81c0b6df34fb3a93d57b6c258e3e9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:17.683810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add81c0b6df34fb3a93d57b6c258e3e9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:17.684687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add81c0b6df34fb3a93d57b6c258e3e9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:17.700843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:17.701663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.702034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.702366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.702724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add81c0b6df34fb3a93d57b6c258e3e9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:17.713632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:17.722220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c677ef2-a52e-4128-a508-8011f189945e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add81c0b6df34fb3a93d57b6c258e3e9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:17.767960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 829/1633] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:17 2022] DELETE /identity/v3/users/add81c0b6df34fb3a93d57b6c258e3e9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:26:17.775537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.775537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.775537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.792069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:17.792069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d869ada-2ea6-4c13-867d-370ade828378 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:17.808776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:17.811921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:17.844543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 805/1634] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:26:17.865989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12f11-b959-4e36-b236-abc0f0bd7073 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.866264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12f11-b959-4e36-b236-abc0f0bd7073 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.866573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e12f11-b959-4e36-b236-abc0f0bd7073 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.868260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 806/1635] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jun 27 15:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:26:17.882080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:17.882080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:17.882425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:17.921065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:17.922366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:17.986224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 830/1636] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:18.063318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:18.065392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:18.086867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:18.087947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 807/1637] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:26:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:26:18.110530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:18.114143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:18.222228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f23954e-8121-4dc7-9eb8-269bd6c266d7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:18.231508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:18.233233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:18.233777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:18.234293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:18.253032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:18.254780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:18.270991 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 831/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:26:18.301891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802ff76-1b41-4aa1-8558-a43bf78110d4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:18.310771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:18.311467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:18.311851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:18.312052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:18.353321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:18.368567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:18.413839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 808/1639] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:21.968020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:21.973727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:22.072210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53756902-b10b-4f82-9f1d-a12b53c6973d service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:22.102781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:22.105271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:22.107659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:22.108746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] PATH_INFO: `/v3/users/b952197a0c3e417eb6fe206f2954743f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:22.115602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] RBAC: Authorizing `identity:delete_user(user_id=b952197a0c3e417eb6fe206f2954743f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:22.121858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:22.166844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:22.169549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:22.276046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ba6b5-f375-47df-b7a9-2cea11fa28e3 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:22.308774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:22.310701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:22.318204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:22.318798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] PATH_INFO: `/v3/projects/b97d6836a163455b8e1b87eef3cbebe1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:22.326308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97d6836a163455b8e1b87eef3cbebe1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:22.338949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:22.351758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b952197a0c3e417eb6fe206f2954743f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.353822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b952197a0c3e417eb6fe206f2954743f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.359408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b952197a0c3e417eb6fe206f2954743f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.379075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b952197a0c3e417eb6fe206f2954743f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.402937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-767d9cc8-d7d9-4a81-af9c-e2491beafced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b952197a0c3e417eb6fe206f2954743f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.480002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 832/1640] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:21 2022] DELETE /identity/v3/users/b952197a0c3e417eb6fe206f2954743f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:22.487398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:22.487398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:22.487898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:22.555943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97d6836a163455b8e1b87eef3cbebe1', 'request_id': 'req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.574600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:22.577306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:22.583549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97d6836a163455b8e1b87eef3cbebe1', 'request_id': 'req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.625532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97d6836a163455b8e1b87eef3cbebe1', 'request_id': 'req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.681062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] The token cache is being invalidate because project b97d6836a163455b8e1b87eef3cbebe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:22.681599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.683882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:22.702266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 809/1641] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:22 2022] DELETE /identity/v3/projects/b97d6836a163455b8e1b87eef3cbebe1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:22.706890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:22.710223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:22.723639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:22.908279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 833/1642] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:22.961058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d241-4d6c-4581-a67c-ac83ac1f6cef admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:22.977814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:22.978883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:22.979284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:22.979676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] PATH_INFO: `/v3/users/c75421fe678a4895a4e8bc1adb4e42e5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:22.989424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75421fe678a4895a4e8bc1adb4e42e5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:22.990703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:22.990703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Jun 27 15:26:22.990703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: Jun 27 15:26:22.992270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:23.160161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75421fe678a4895a4e8bc1adb4e42e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:23.229716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75421fe678a4895a4e8bc1adb4e42e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:23.341094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75421fe678a4895a4e8bc1adb4e42e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:23.343955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75421fe678a4895a4e8bc1adb4e42e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:23.345619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75421fe678a4895a4e8bc1adb4e42e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:23.408822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 810/1643] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:22 2022] DELETE /identity/v3/users/c75421fe678a4895a4e8bc1adb4e42e5 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:23.418533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:23.421400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:23.470008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:23.561822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e625ca-b871-4412-a159-5b2ac5431e99 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:23.704668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:23.705396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:23.705744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:23.706209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:23.713258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:23.955438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:23.976290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:24.024430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 834/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:27.833571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:27.835403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:27.869060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39b021-0e03-459b-9a01-39f5fc46e10b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:27.885365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:27.886375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:27.886659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:27.886933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] PATH_INFO: `/v3/projects/074c7de8debf489aa5c073a1ae623259` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:27.888624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=074c7de8debf489aa5c073a1ae623259)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:27.890907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:27.962442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:27.964061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:27.999083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a8866-016e-41a2-b711-0c1e9b49a19c service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:28.027752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:28.028714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.029003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.029466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] PATH_INFO: `/v3/projects/861118cee4ba49878a27ab313b4f1cba` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.035107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861118cee4ba49878a27ab313b4f1cba)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:28.037039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:26:28.037039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jun 27 15:26:28.037039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: Jun 27 15:26:28.042236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:28.102425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074c7de8debf489aa5c073a1ae623259', 'request_id': 'req-5b1a1f52-72cb-4019-8fc5-2223279002c1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.140306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074c7de8debf489aa5c073a1ae623259', 'request_id': 'req-5b1a1f52-72cb-4019-8fc5-2223279002c1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.155739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074c7de8debf489aa5c073a1ae623259', 'request_id': 'req-5b1a1f52-72cb-4019-8fc5-2223279002c1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.210214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] The token cache is being invalidate because project 074c7de8debf489aa5c073a1ae623259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:28.210521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.211304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5b1a1f52-72cb-4019-8fc5-2223279002c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.226043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 811/1645] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:27 2022] DELETE /identity/v3/projects/074c7de8debf489aa5c073a1ae623259 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:28.226863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861118cee4ba49878a27ab313b4f1cba', 'request_id': 'req-c2b52b59-6d58-4b35-abf0-01703f446298'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.252831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861118cee4ba49878a27ab313b4f1cba', 'request_id': 'req-c2b52b59-6d58-4b35-abf0-01703f446298'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.268021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2d7f-6297-4d23-98f4-efe597da5499 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.268441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2d7f-6297-4d23-98f4-efe597da5499 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.269294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2d7f-6297-4d23-98f4-efe597da5499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.290984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861118cee4ba49878a27ab313b4f1cba', 'request_id': 'req-c2b52b59-6d58-4b35-abf0-01703f446298'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.319324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:28.325365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] The token cache is being invalidate because project 861118cee4ba49878a27ab313b4f1cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:28.326016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.327737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c2b52b59-6d58-4b35-abf0-01703f446298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:28.350937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 835/1646] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:27 2022] DELETE /identity/v3/projects/861118cee4ba49878a27ab313b4f1cba => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:26:28.352652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.352945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.353222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.355938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-de7b2d7f-6297-4d23-98f4-efe597da5499 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.357160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-de7b2d7f-6297-4d23-98f4-efe597da5499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.428048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:28.466218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] MFA Rules not processed for user `5e2597ff867544f9953c519bf29a9bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.469783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.556475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 812/1647] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.558498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b3b0-fe03-4bab-8506-aa26ec363ab6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.558498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b3b0-fe03-4bab-8506-aa26ec363ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.558805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b3b0-fe03-4bab-8506-aa26ec363ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.626551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e5e5b3b0-fe03-4bab-8506-aa26ec363ab6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.634007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e5e5b3b0-fe03-4bab-8506-aa26ec363ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.681334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 813/1648] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.683506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dd55e-e9e1-47d2-a8c6-141a4b76236c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.683840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dd55e-e9e1-47d2-a8c6-141a4b76236c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.684125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dd55e-e9e1-47d2-a8c6-141a4b76236c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.708926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7d5dd55e-e9e1-47d2-a8c6-141a4b76236c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.710228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7d5dd55e-e9e1-47d2-a8c6-141a4b76236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.724420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 814/1649] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.725288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d76e4-2fcc-41a9-bff1-469cfbb95d06 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.725692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d76e4-2fcc-41a9-bff1-469cfbb95d06 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.725980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d76e4-2fcc-41a9-bff1-469cfbb95d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.740014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 836/1650] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.742109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:28.759291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-068d76e4-2fcc-41a9-bff1-469cfbb95d06 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.759291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-068d76e4-2fcc-41a9-bff1-469cfbb95d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.768419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 815/1651] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.770367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.770759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.771139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.794912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312fed0-e32f-4012-8d1a-f33158233433 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:28.795109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] MFA Rules not processed for user `5e2597ff867544f9953c519bf29a9bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:28.796646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.813178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:28.813178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.813178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.813178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.814560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:28.814560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:28.817260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:28.831288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 816/1652] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:28.841292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:28.866520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8bd6d-2b6b-40ce-be34-320af158d71b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:28.877158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 837/1653] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:28.878670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:28.882769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:28.884590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.885578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.886549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.888797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:28.891293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:28.891482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:28.902477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:28.930503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 817/1654] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:28.931655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:28.948704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a1f1e-3dd3-4e99-a445-15fff8f153e3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:28.964097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:28.965119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:28.965639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:28.966051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:28.966932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:28.977464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:29.062211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 818/1655] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:28 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:29.062211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:29.066022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:29.108811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13678d78-c131-4074-b5de-3be07e59b192 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:29.244814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c33105-f5d4-411c-aaaa-b78040ecb880 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:29.262010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:29.262946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:29.263231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:29.263509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] PATH_INFO: `/v3/users/d32af5ae84b34f079b84bc416fb5b576` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:29.267519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32af5ae84b34f079b84bc416fb5b576)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:29.271922 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:29.283845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:29.284780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:29.285062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:29.285347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:29.355525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:29.409374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32af5ae84b34f079b84bc416fb5b576'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:29.411327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:29.443309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32af5ae84b34f079b84bc416fb5b576'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:29.480687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32af5ae84b34f079b84bc416fb5b576'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:29.482067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32af5ae84b34f079b84bc416fb5b576'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:29.483143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32af5ae84b34f079b84bc416fb5b576'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:29.511609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 819/1656] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:29 2022] DELETE /identity/v3/users/d32af5ae84b34f079b84bc416fb5b576 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:29.513359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:29.513559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:29.515668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:29.518162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:29.553963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 838/1657] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:29.556862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:29.619389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443f6e6-5cfe-40af-9899-46e9df39d01d service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:29.796735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b7cd6-7fb8-4e12-8ac7-90f82145f53a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:29.817847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:29.820261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:29.820985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:29.821549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:29.822472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:29.840142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8690fe6-c593-4881-988f-a1060927d1b2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:29.913139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:29.914066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:29.914184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:29.914396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:29.915127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:29.917267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:29.957516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 820/1658] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:29 2022] POST /identity/v3/projects => generated 389 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:29.960729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:29.961382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:29.962507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:29.985168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 839/1659] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:29 2022] POST /identity/v3/users => generated 287 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:29.986768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.015828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dceef-f309-4ce6-b794-9170fcf27007 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.027218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:30.029206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:30.229325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 821/1660] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:30.235895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.240826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:30.245007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.249804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.253473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.262223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.263934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.266365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.271442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d64b5e-72d4-418b-970d-6af4dd9cea4c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.285004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.285953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.286351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.286739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.287642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.293927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.319811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 822/1661] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:30.325474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.333705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 840/1662] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:29 2022] POST /identity/v3/users => generated 295 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:30.334436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:30.337325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.359282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf4eb9-2fa6-4a63-9af9-7e60dcfe055e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.370627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.371635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.372058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.372511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.373451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.375370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.432875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 841/1663] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:30.434188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.463245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cdded-89c3-42ce-a798-22daea3cffd8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.468982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:30.482556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.491182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.492066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.492593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.493741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.496201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.521366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6160190-7bf1-4fb9-bd6d-3f6fab57627d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.529962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.531229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.531805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.533085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.535632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 842/1664] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:30.537006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.562983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3da05-d72a-47a2-9713-4a1059f550ce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.565061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.567825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.580735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.581876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.582243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.582524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] PATH_INFO: `/v3/projects/eb060f4f653f4c3194430b876587d6a4/users/f870bc0a641942c6be2917f8f83ba1fd/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.598336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 823/1665] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:30.599453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.622227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279befef-1d5e-4232-acae-0944e8777e88 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.623591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb060f4f653f4c3194430b876587d6a4, user_id=f870bc0a641942c6be2917f8f83ba1fd, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.628298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133fab1-b231-4edc-a893-3209874e2f71 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.630176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.631037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.631431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.631835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.632739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.634154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:30.636211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.660062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 824/1666] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:30.665121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.667979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 843/1667] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:30 2022] PUT /identity/v3/projects/eb060f4f653f4c3194430b876587d6a4/users/f870bc0a641942c6be2917f8f83ba1fd/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:30.678920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.679241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.679473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.713037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35821-7592-4b1f-8cb1-122b54dce7f0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:30.754300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] MFA Rules not processed for user `f870bc0a641942c6be2917f8f83ba1fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:30.756015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:30.880981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:30.882373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.882664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.883257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] PATH_INFO: `/v3/projects/b9062a69feb844c58ccb2257100048de/users/54e0137e78b448ca9270e06eeb7048a6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:30.923624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9062a69feb844c58ccb2257100048de, user_id=54e0137e78b448ca9270e06eeb7048a6, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:30.927580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95c58a-c0fa-4f49-8f9e-4d83c3efb653 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:30.960069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 844/1668] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:30.963092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:30.973238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 825/1669] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:30 2022] PUT /identity/v3/projects/b9062a69feb844c58ccb2257100048de/users/54e0137e78b448ca9270e06eeb7048a6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:26:30.974964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:30.975255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:30.975542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:31.012302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e1f6-b2e0-4e15-b3c9-284f31400c7d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:31.070697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] MFA Rules not processed for user `54e0137e78b448ca9270e06eeb7048a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:31.076903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:31.200831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:31.202593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:31.203357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:31.204119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:31.210835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:31.216173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:31.298215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 845/1670] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:30 2022] POST /identity/v3/projects => generated 371 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:31.302360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:31.628398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8010a4-e794-4b84-992f-1d9aa1555a62 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:31.655015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:31.657040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:31.659979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:31.659979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:31.666223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:31.676557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:31.738410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:31.756643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 846/1671] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:31 2022] POST /identity/v3/users => generated 292 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:31.763694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:31.797023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e72d22-37ea-4e25-b12f-5bf5c0699231 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:31.814490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 826/1672] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 840 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:31.818253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:31.819602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:31.822183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:31.823073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:31.825709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:31.852839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:31.875153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:31.911148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2151a-e6bc-4925-b379-e4e821368e5c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:31.931639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 847/1673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:31.933077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:31.934223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:31.935025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:31.935503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:31.936960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:31.938202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:31.951801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f83f2-e2bc-4b4b-8d9c-1a9d92a01116 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:31.982319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbdfd0-304d-4b47-83c8-99d0039d628e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:32.012757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:32.022921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.023970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.024644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] PATH_INFO: `/v3/projects/3efd4a142f15430dad4acdd4c48e7d28/users/fc726d5ca3804482873d6b261b919a1b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.049477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 827/1674] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:31 2022] POST /identity/v3/projects => generated 389 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:32.051474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.051987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.052514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.075025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efd4a142f15430dad4acdd4c48e7d28, user_id=fc726d5ca3804482873d6b261b919a1b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:32.099287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:32.144664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 848/1675] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:31 2022] PUT /identity/v3/projects/3efd4a142f15430dad4acdd4c48e7d28/users/fc726d5ca3804482873d6b261b919a1b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:32.146168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:32.150560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] MFA Rules not processed for user `effe0200dc584f11923e83ed240138ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:32.152820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:32.191860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:32.386960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 828/1676] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:32.388677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:32.567980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0e5b-3b0f-4e9e-8155-69d4bf51bac5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:32.594738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:32.596637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.597441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.598114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.599709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:32.618570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2d3cc-4d95-4ec1-9bde-4d09790d9c71 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:32.694535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068c03c-fe06-4203-a86c-20f22480b7dd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:32.719279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 849/1677] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:32 2022] POST /identity/v3/users => generated 301 bytes in 574 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:32.721326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:32.722767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.722952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.723291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.723742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.725173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.727751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.734279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:32.743759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b035a-4f88-4d0b-b760-773c22407e49 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:32.798872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] MFA Rules not processed for user `effe0200dc584f11923e83ed240138ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:32.804612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 829/1678] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:32.807438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:32.815324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.815829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.816359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.839461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 830/1679] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jun 27 15:26:32 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:26:32.841560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:32.843737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:32.849266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 850/1680] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:32.849266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:32.854207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:32.869620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890444f8-c7e1-407a-bad7-2b11fe7e9c0a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:32.875152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:32.876439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.876881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.877303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] PATH_INFO: `/v3/users/fecf8c076b2e4553aba77b7eef4296a7` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.879301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fecf8c076b2e4553aba77b7eef4296a7)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:32.879777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563c05e-7c29-432e-a641-46f4b54d8337 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:32.881963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:32.888272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:32.889872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:32.890529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:32.891012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] PATH_INFO: `/v3/projects/e9c75d75a0ea45818f28071163cb93ad` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:32.894974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c75d75a0ea45818f28071163cb93ad)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:32.899247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:33.007958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fecf8c076b2e4553aba77b7eef4296a7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.026757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fecf8c076b2e4553aba77b7eef4296a7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.046490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c75d75a0ea45818f28071163cb93ad', 'request_id': 'req-2e24586c-cf07-418d-82ee-21cd52391f15'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.059498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fecf8c076b2e4553aba77b7eef4296a7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.061382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fecf8c076b2e4553aba77b7eef4296a7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.064208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fecf8c076b2e4553aba77b7eef4296a7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.069331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c75d75a0ea45818f28071163cb93ad', 'request_id': 'req-2e24586c-cf07-418d-82ee-21cd52391f15'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.086602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c75d75a0ea45818f28071163cb93ad', 'request_id': 'req-2e24586c-cf07-418d-82ee-21cd52391f15'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.099915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 831/1681] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:32 2022] DELETE /identity/v3/users/fecf8c076b2e4553aba77b7eef4296a7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:33.101157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:33.105830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.132250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] The token cache is being invalidate because project e9c75d75a0ea45818f28071163cb93ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:33.132250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.132250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:33.139275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 851/1682] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:32 2022] DELETE /identity/v3/projects/e9c75d75a0ea45818f28071163cb93ad => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:33.139275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:33.139275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.158427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:33.265460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:33.307550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41fb76-4990-4a22-ae0b-5c4ed9ef3fdb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:33.323984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:33.324725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.325007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.325285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.326063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:33.327792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f49ae-6ac6-4589-ba1a-7f88a8b70653 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:33.347826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24586c-cf07-418d-82ee-21cd52391f15 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:33.352916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 832/1683] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:33.352916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688fcf-c13e-4a47-af82-56276a35a31f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.352916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688fcf-c13e-4a47-af82-56276a35a31f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.352916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6688fcf-c13e-4a47-af82-56276a35a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.364415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:33.366299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.366792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.367290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] PATH_INFO: `/v3/projects/3efd4a142f15430dad4acdd4c48e7d28/users/fc726d5ca3804482873d6b261b919a1b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.372287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efd4a142f15430dad4acdd4c48e7d28, user_id=fc726d5ca3804482873d6b261b919a1b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:33.378588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:33.409811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 852/1684] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:33 2022] PUT /identity/v3/projects/3efd4a142f15430dad4acdd4c48e7d28/users/fc726d5ca3804482873d6b261b919a1b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:33.411587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:33.417932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.457158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e6688fcf-c13e-4a47-af82-56276a35a31f None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:33.458548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e6688fcf-c13e-4a47-af82-56276a35a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.591365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 833/1685] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:33.592546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099f6e-2af4-4e75-8d23-ce564e342f1a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.592883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099f6e-2af4-4e75-8d23-ce564e342f1a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.593161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099f6e-2af4-4e75-8d23-ce564e342f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.651680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cd099f6e-2af4-4e75-8d23-ce564e342f1a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:33.659963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cd099f6e-2af4-4e75-8d23-ce564e342f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.699302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcd5ba-e694-4014-ac92-b3126053824f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:33.713689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:33.715264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.717389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.718603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.725478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.835253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 834/1686] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:33.838982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:33.841035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:33.850798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.852919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.853544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.856387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 853/1687] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:33.857598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:33.859799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.878911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070c18-5665-4e5a-a4e6-8b0434f26da7 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:33.885059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:33.888135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.888689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.889338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] PATH_INFO: `/v3/projects/1b97d142ddc3401d998421f25dd2e2d7/users/9db8f3e113024e3f9b4f0b2cc27d7399/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:33.940800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b97d142ddc3401d998421f25dd2e2d7, user_id=9db8f3e113024e3f9b4f0b2cc27d7399, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:33.948065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ad244-c538-4111-a3a6-b5e28c453b0b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:33.969205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:33.977777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:33.983783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 854/1688] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:33 2022] PUT /identity/v3/projects/1b97d142ddc3401d998421f25dd2e2d7/users/9db8f3e113024e3f9b4f0b2cc27d7399/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:33.986101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:33.988571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:33.989035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.113435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] MFA Rules not processed for user `fc726d5ca3804482873d6b261b919a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:34.122426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:34.205569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:34.273305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:34.288732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 835/1689] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:34.290111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:34.293158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:34.448727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 855/1690] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:34.451524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:34.454624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:34.537416 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc364f1-12a0-43c5-8c66-b8c186fd01a3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:34.553320 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:34.554661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:34.554985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:34.555281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.589867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:34.591810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:34.651496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 836/1691] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:34.655803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:34.713025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554625d8-2009-48a2-82d8-3a3f6de55edf service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:34.730280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a536b6a-bb68-49ce-a0f1-9afb06d5c10e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:34.732235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:34.732980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:34.733268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:34.733548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.734321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:34.743344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1c63-0603-48fb-9d75-a1ea7b593465 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:34.749738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:34.752855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:34.752855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:34.758502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.798179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 837/1692] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:34.798482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:34.798642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:34.802247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:34.802404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.802918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:34.833177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 856/1693] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:34.834506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:34.843939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:34.870066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:34.873758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:34.926741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 838/1694] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:34.929803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:34.964966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c2152-8bad-4f45-80fa-30975af63d2e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:34.982504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:34.985452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:34.985740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:34.986020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] PATH_INFO: `/v3/projects/1b97d142ddc3401d998421f25dd2e2d7/users/9db8f3e113024e3f9b4f0b2cc27d7399/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:34.993793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b97d142ddc3401d998421f25dd2e2d7, user_id=9db8f3e113024e3f9b4f0b2cc27d7399, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:35.004154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:35.033447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de76b6d-7b2b-4d97-8e7f-5b7651bece26 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:35.033892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 839/1695] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:34 2022] PUT /identity/v3/projects/1b97d142ddc3401d998421f25dd2e2d7/users/9db8f3e113024e3f9b4f0b2cc27d7399/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:35.035659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:35.257629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:35.258783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:35.259275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:35.259746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:35.307249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb0fc4-2ebf-411c-823a-f9a4b3f7923c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:35.351808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:35.353042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:35.353442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:35.353773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:35.355198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:35.355774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:35.380865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded7721-6004-41c1-bd41-7f9a9c3239b7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:35.462906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 840/1696] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:35.465420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0bd7e-d9c9-47e5-aed5-456d1ba425f8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:35.465902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0bd7e-d9c9-47e5-aed5-456d1ba425f8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:35.466396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0bd7e-d9c9-47e5-aed5-456d1ba425f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:35.517394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:35.639779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e3a0bd7e-d9c9-47e5-aed5-456d1ba425f8 None None] MFA Rules not processed for user `9db8f3e113024e3f9b4f0b2cc27d7399`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:35.641872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e3a0bd7e-d9c9-47e5-aed5-456d1ba425f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:35.691260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:35.707019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:35.755223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 857/1697] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 921 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:35.757521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:35.804532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8594d-e416-43d5-823e-4092d25e20d1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:35.812580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:35.814180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:35.814715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:35.815896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:35.817529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:35.821486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:35.883793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 858/1698] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:35 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:35.907148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:35.967415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:36.113519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 841/1699] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 649 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:36.115880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56f9f-0f7c-4aed-8cc1-84b19e07a4bf None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:36.116362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56f9f-0f7c-4aed-8cc1-84b19e07a4bf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:36.116866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff56f9f-0f7c-4aed-8cc1-84b19e07a4bf None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:36.124445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 842/1700] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jun 27 15:26:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:26:36.152237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1c7d-6b7a-489f-a54b-23bbb1f82503 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:36.160072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff56f9f-0f7c-4aed-8cc1-84b19e07a4bf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:36.162162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:36.163535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:36.164200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:36.164200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:36.172478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:36.172478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:36.236608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 859/1701] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:35 2022] POST /identity/v3/users => generated 302 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:36.240948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:36.247667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:36.545414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c505e3-a82b-43bf-abfc-18c6af878af2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:36.551494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:36.552311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:36.552737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:36.553121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:36.614277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff56f9f-0f7c-4aed-8cc1-84b19e07a4bf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:36.620089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:36.621341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:36.621873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:36.622392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:36.634804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:36.749944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:36.782953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:36.786191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ad28-e885-441e-a949-904b2bdb0a43 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:36.809386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 860/1702] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:36.812614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618f-fd38-4383-9ac5-2eb1199f395b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:36.813346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618f-fd38-4383-9ac5-2eb1199f395b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:36.814014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c618f-fd38-4383-9ac5-2eb1199f395b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:36.855157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-215c618f-fd38-4383-9ac5-2eb1199f395b None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:36.858431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-215c618f-fd38-4383-9ac5-2eb1199f395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:36.969190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:36.972851 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.009147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 843/1703] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 884 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:37.010322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.048432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 861/1704] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:37.058850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6936ab-803f-47e3-9d17-11baa0d287a8 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.061953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.062360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.062774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.067589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 862/1705] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jun 27 15:26:37 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:26:37.068605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.074636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.076028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.077005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.077464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.077898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.078825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.080926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.103302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 844/1706] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:37.107921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.131705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063c285-7e38-4ad9-be24-6e1195816acf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.136636 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85eeb4-0720-4080-80f6-9d7856e33655 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.151798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.151798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.151798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.151798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] PATH_INFO: `/v3/projects/090e27bd88cc4d91a6e06fc8a93d56eb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.153582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=090e27bd88cc4d91a6e06fc8a93d56eb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.153713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.154923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.155099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.155233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.155233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.186203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.187908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024949ab-43d3-472f-b59d-8f57f662373f service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.217490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 845/1707] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:37.219105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.219407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.219695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.268655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:37.270608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.337192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090e27bd88cc4d91a6e06fc8a93d56eb', 'request_id': 'req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.362524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090e27bd88cc4d91a6e06fc8a93d56eb', 'request_id': 'req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.382067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090e27bd88cc4d91a6e06fc8a93d56eb', 'request_id': 'req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.425724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] The token cache is being invalidate because project 090e27bd88cc4d91a6e06fc8a93d56eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:37.426045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.430929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.433157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 863/1708] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:37 2022] DELETE /identity/v3/projects/090e27bd88cc4d91a6e06fc8a93d56eb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:37.440170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.452564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.480793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 846/1709] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jun 27 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:37.481861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.483357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.549276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:37.691964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fcc4-9a8f-4265-8e47-9ba69365b1ff admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.717935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.721428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.725245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.725783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.727969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.729769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdaf5a-a93f-49fc-9991-3fe608b51a4b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.742236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650167bd-a21b-4379-934d-8b51c3f2423b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.759539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.760729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.766272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.766796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] PATH_INFO: `/v3/users/54e0137e78b448ca9270e06eeb7048a6` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.771996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54e0137e78b448ca9270e06eeb7048a6)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.776290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.786992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 864/1710] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:37.787814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.787814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.788058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.853923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:37.869595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.899001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 865/1711] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:37.899750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.902795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54e0137e78b448ca9270e06eeb7048a6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.913517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54e0137e78b448ca9270e06eeb7048a6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.929768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fd6bc-b8b3-44ab-a82e-69a1e408d947 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:37.932336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54e0137e78b448ca9270e06eeb7048a6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.933747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54e0137e78b448ca9270e06eeb7048a6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.935187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54e0137e78b448ca9270e06eeb7048a6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:37.940918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:37.941645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:37.941932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:37.942728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:37.946997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.969918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 847/1712] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:37 2022] DELETE /identity/v3/users/54e0137e78b448ca9270e06eeb7048a6 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:37.974879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:37.978282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:37.981012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:37.985955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6d4a9-9833-42e4-a3cd-09a03f8e80c6 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:37.995795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:38.014980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 866/1713] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:38.017806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8abf-b58b-455e-bd83-d94a1bd745ba None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.018224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8abf-b58b-455e-bd83-d94a1bd745ba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.018425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf8abf-b58b-455e-bd83-d94a1bd745ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.114552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f4bf8abf-b58b-455e-bd83-d94a1bd745ba None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:38.114552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f4bf8abf-b58b-455e-bd83-d94a1bd745ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:38.259643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbfd1d-61d3-4849-8333-5b6c8429545e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:38.282205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:38.284701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.285020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.285309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] PATH_INFO: `/v3/projects/709016930c9e4fa1842675ff08402a2f/users/e228ccc81ae64f6a8f9878d019c155ef/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.339235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] RBAC: Authorizing `identity:create_grant(project_id=709016930c9e4fa1842675ff08402a2f, user_id=e228ccc81ae64f6a8f9878d019c155ef, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:38.345912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:38.371955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 867/1714] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:38.376665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2f71-9d6d-46b6-8733-2016f34e77ed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.377161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2f71-9d6d-46b6-8733-2016f34e77ed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.377575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a2f71-9d6d-46b6-8733-2016f34e77ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.416668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 848/1715] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:37 2022] PUT /identity/v3/projects/709016930c9e4fa1842675ff08402a2f/users/e228ccc81ae64f6a8f9878d019c155ef/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:38.419329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:38.465636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-496a2f71-9d6d-46b6-8733-2016f34e77ed None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:38.467074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-496a2f71-9d6d-46b6-8733-2016f34e77ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:38.524924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:38.571033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 868/1716] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:38.583995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.584392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.584795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.624901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] MFA Rules not processed for user `e228ccc81ae64f6a8f9878d019c155ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:38.626878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:38.721956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510dcd-ce52-4435-acea-9c317875adab admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:38.751261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:38.753194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.756825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.757498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.758581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:38.759081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:38.772214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:38.821973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 849/1717] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:38.821973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:38.826514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:38.929280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 869/1718] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:38.933937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:38.965447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f219da-255d-4688-9613-95a44ed4b507 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:38.970208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:38.971440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:38.973962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:38.974774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:38.984301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503fe34-8e23-4540-89cc-7542e6674f8d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.003172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.005897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.029009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.031886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 850/1719] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:39.033483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.049545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b87d4-5a3d-4e74-b111-9a90c06f9f7e service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.052515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.053155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.059048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.063197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.069124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.069809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.070103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.070381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.081000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.082314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.084410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.169696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 870/1720] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:38 2022] POST /identity/v3/projects => generated 403 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:39.178808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.186826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 851/1721] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:39 2022] POST /identity/v3/projects => generated 405 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:39.220820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:39.230305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.530181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e0edf-01e5-4ffc-b7a8-71b6762156c1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.618902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2187d3-db84-49c4-9d77-f9cebdc9e034 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.650688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.651602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.653546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.653953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.654838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.662122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.742801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 852/1722] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:39 2022] POST /identity/v3/users => generated 309 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:39.749052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.784607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b655cfd-0592-4910-a7b1-1094831008d5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.802345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.804917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.805850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.806324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.807054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.807746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.808711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.813039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.814600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.816007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.817665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.820448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.859358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 853/1723] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:26:39.866336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.884868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 871/1724] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:39 2022] POST /identity/v3/users => generated 302 bytes in 711 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:39.890269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:39.907525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd8147-97a1-4560-8ef0-4a4350967be1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.927210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.928281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449d368-3705-4a2c-9e97-e954297d13f9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:39.930546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.933032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.933845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] PATH_INFO: `/v3/projects/0499ee310cb34f02a2307b453710a571/users/ab9d49df938348519755ad17ec33a78c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.945823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:39.952449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:39.953631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:39.957475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:39.958420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:39.961435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:39.970896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:39.998491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0499ee310cb34f02a2307b453710a571, user_id=ab9d49df938348519755ad17ec33a78c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:40.005436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:40.015224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 872/1725] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:40.020267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:40.076826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 854/1726] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:39 2022] PUT /identity/v3/projects/0499ee310cb34f02a2307b453710a571/users/ab9d49df938348519755ad17ec33a78c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:40.079857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3ba8c-6406-43a5-87d1-40b85b6b5b11 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:40.082464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:40.360386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b44d3-1946-4f48-9809-248e797a07b0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:40.382226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:40.382226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.382226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.382226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.382925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:40.383193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bb81b-a9a8-4505-ae9e-244b7516f381 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:40.400439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:40.403954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.403954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.403954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.403954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:40.416490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:40.428888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 855/1727] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:40.436203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025937-62f2-4286-b343-b4eee7b85f3d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.437774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025937-62f2-4286-b343-b4eee7b85f3d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.438731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025937-62f2-4286-b343-b4eee7b85f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.457799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 873/1728] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:40.459701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:40.474666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9047cd-2cc2-48d8-b788-08f016667287 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:40.481644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:40.482881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.483467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.484299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] PATH_INFO: `/v3/projects/0499ee310cb34f02a2307b453710a571/users/ab9d49df938348519755ad17ec33a78c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.487600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0499ee310cb34f02a2307b453710a571, user_id=ab9d49df938348519755ad17ec33a78c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:40.490121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9a025937-62f2-4286-b343-b4eee7b85f3d None None] MFA Rules not processed for user `effe0200dc584f11923e83ed240138ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:40.490931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:40.491512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9a025937-62f2-4286-b343-b4eee7b85f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:40.521898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 874/1729] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:40 2022] PUT /identity/v3/projects/0499ee310cb34f02a2307b453710a571/users/ab9d49df938348519755ad17ec33a78c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:40.523190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:40.588501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:40.674385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 856/1730] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:40.676698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1fa71-d25f-4699-88d0-0c8920d82433 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.677183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1fa71-d25f-4699-88d0-0c8920d82433 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.677331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1fa71-d25f-4699-88d0-0c8920d82433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.765652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f3b1fa71-d25f-4699-88d0-0c8920d82433 None None] MFA Rules not processed for user `ab9d49df938348519755ad17ec33a78c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:40.767048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f3b1fa71-d25f-4699-88d0-0c8920d82433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:40.788832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215bb7a-4523-4550-8306-b58fb57b909d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:40.828644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:40.834051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:40.837165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:40.839878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] PATH_INFO: `/v3/projects/5f9abf50927347828b75d5a8763c35ce/users/68c55f4416bc48768f6d043006da2b1e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:40.933361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9abf50927347828b75d5a8763c35ce, user_id=68c55f4416bc48768f6d043006da2b1e, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:40.941552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:40.997252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 875/1731] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:40 2022] PUT /identity/v3/projects/5f9abf50927347828b75d5a8763c35ce/users/68c55f4416bc48768f6d043006da2b1e/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:41.000110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:41.003688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:41.102169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 857/1732] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:41.103461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:41.103751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:41.115574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:41.191547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:41.194890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:41.204349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705d4d-2677-4ecb-aaef-1c292e9d7447 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:41.217713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:41.225716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:41.226246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:41.226911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] PATH_INFO: `/v3/projects/b9062a69feb844c58ccb2257100048de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:41.230023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9062a69feb844c58ccb2257100048de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:41.234742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:41.252658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 858/1733] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:41.254049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:41.264289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:41.281123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:41.406773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9062a69feb844c58ccb2257100048de', 'request_id': 'req-0710620e-9684-4e20-8089-7e083afc97c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:41.420041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9062a69feb844c58ccb2257100048de', 'request_id': 'req-0710620e-9684-4e20-8089-7e083afc97c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:41.441710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9062a69feb844c58ccb2257100048de', 'request_id': 'req-0710620e-9684-4e20-8089-7e083afc97c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:41.492092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] The token cache is being invalidate because project b9062a69feb844c58ccb2257100048de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:41.492789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:41.496691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:41.500638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 876/1734] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:40 2022] DELETE /identity/v3/projects/b9062a69feb844c58ccb2257100048de => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:41.501915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:41.508120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:41.515795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad129644-b172-4549-a0a4-262b7d623cba None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:41.658243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:41.784057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:41.785246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:41.785721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:41.786160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:41.806326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:41.828292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710620e-9684-4e20-8089-7e083afc97c9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:41.869388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:41.877683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:41.897228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:41.898415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:41.910630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:42.128731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:42.138022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:42.187125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 859/1735] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:42.189182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:42.195357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:42.234653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:42.239537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630b2aa-fbad-4e64-9d8c-8e5eea7a3ec4 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:42.287392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 877/1736] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:42.287942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488a8e4-698e-457b-9470-c1a702f6a743 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:42.293561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:42.294084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:42.294601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:42.306390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:42.307462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:42.307808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:42.308163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] PATH_INFO: `/v3/users/9db8f3e113024e3f9b4f0b2cc27d7399` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:42.310831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db8f3e113024e3f9b4f0b2cc27d7399)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:42.314018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:42.425850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] MFA Rules not processed for user `995fa1760f5241588a493e5f894b0e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:42.427106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:42.535668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db8f3e113024e3f9b4f0b2cc27d7399'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:42.612921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db8f3e113024e3f9b4f0b2cc27d7399'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:42.657950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:42.725468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db8f3e113024e3f9b4f0b2cc27d7399'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:42.725647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 878/1737] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:42.733061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:42.737661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:42.745114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db8f3e113024e3f9b4f0b2cc27d7399'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:42.748575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6412be6-f10e-48b8-993f-e615cf910988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db8f3e113024e3f9b4f0b2cc27d7399'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:42.811746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25961f92-b8e1-45e1-9bce-52d2c780f5c3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:42.826274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 860/1738] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:42 2022] DELETE /identity/v3/users/9db8f3e113024e3f9b4f0b2cc27d7399 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:42.828801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:42.829294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:42.829782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:43.011098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] MFA Rules not processed for user `e9dab30db0a44655828384db53df29f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:43.026378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:43.120860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:43.122999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:43.123466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:43.123914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] PATH_INFO: `/v3/users/e228ccc81ae64f6a8f9878d019c155ef` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:43.127607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e228ccc81ae64f6a8f9878d019c155ef)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:43.136368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:43.198363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:43.453287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e228ccc81ae64f6a8f9878d019c155ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:43.453287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e228ccc81ae64f6a8f9878d019c155ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:43.453287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e228ccc81ae64f6a8f9878d019c155ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:43.495716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e228ccc81ae64f6a8f9878d019c155ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:43.526849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-085e6f00-efc7-4c25-939c-135255e96723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e228ccc81ae64f6a8f9878d019c155ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:43.565902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 861/1739] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 739 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:43.567394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:43.571997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:43.643400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 879/1740] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:42 2022] DELETE /identity/v3/users/e228ccc81ae64f6a8f9878d019c155ef => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:43.646177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:43.661640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:43.663090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:43.746038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:43.747530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:43.870806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 880/1741] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:43.884022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:43.886074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:44.046769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ff27b-45a8-4319-a51d-9c129c64cbff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:44.068130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:44.070394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:44.074285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:44.076494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:44.122092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:44.132391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:44.188728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 881/1742] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:44.209120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dd6e1-25ce-4552-bf54-4b43f4a64054 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:44.477495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:44.479615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:44.481385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:44.484024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:44.498461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:44.526002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:44.681619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:44.686987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:44.735392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 862/1743] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 1166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:44.816916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:44.819456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:44.821245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:44.824575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:44.922137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384cdd7-7779-40bb-b237-c39d67d758c4 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:44.946976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08f5b-9674-46d9-b6b7-fc5d9512094f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:44.955771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:44.960284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:44.960853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:44.963482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] PATH_INFO: `/v3/projects/1b97d142ddc3401d998421f25dd2e2d7` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:44.970612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b97d142ddc3401d998421f25dd2e2d7)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:44.977806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:45.006031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:45.007223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:45.007694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:45.008546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:45.146153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b97d142ddc3401d998421f25dd2e2d7', 'request_id': 'req-195f9e71-96f6-40c2-8a77-a4d29da71d77'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:45.175341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b97d142ddc3401d998421f25dd2e2d7', 'request_id': 'req-195f9e71-96f6-40c2-8a77-a4d29da71d77'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:45.180310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:45.205784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b97d142ddc3401d998421f25dd2e2d7', 'request_id': 'req-195f9e71-96f6-40c2-8a77-a4d29da71d77'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:45.234288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] The token cache is being invalidate because project 1b97d142ddc3401d998421f25dd2e2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:45.236537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:45.238467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-195f9e71-96f6-40c2-8a77-a4d29da71d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:45.241962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 863/1744] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:44 2022] DELETE /identity/v3/projects/1b97d142ddc3401d998421f25dd2e2d7 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:45.249979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:45.250278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:45.250700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:45.302009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:45.304408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599e08-1b3f-420c-856d-c30c900c702a service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:45.329877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 882/1745] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:45.347833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:45.350210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:45.695299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 864/1746] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:26:45.717964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:45.718360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:45.718501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:45.855566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:45.869359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:45.943091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 883/1747] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:45.950098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:45.969276 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e75ef3-4d13-403a-b6b3-c064a6697bd4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.001513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.002771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.006317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.006932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.008032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:46.008790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.013020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940494aa-4eba-496a-9748-6e8ce286a44f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:46.051995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:46.065961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7807208-e3d4-4198-8633-545d560f5abf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.066650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 865/1748] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:46.079091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.081268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.083307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.105279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.106520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.107491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.108664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.111618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.116250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:46.168607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] MFA Rules not processed for user `68c55f4416bc48768f6d043006da2b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:46.170354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:46.178675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:46.208213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 884/1749] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:46 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:26:46.217377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:46.369180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 866/1750] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:46.369180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:46.369180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:46.383181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3fa7b-eec1-4237-ac98-07d46b81a8ae None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.470365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.472014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.472758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.473072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] PATH_INFO: `/v3/projects/709016930c9e4fa1842675ff08402a2f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.475046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709016930c9e4fa1842675ff08402a2f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.478900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:46.508915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d816a83-9c86-41e2-aafc-3c1b9483c76f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.516395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.517760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.518397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.518819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.523370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.534382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:46.589842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 885/1751] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:46 2022] POST /identity/v3/users => generated 290 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:46.591326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:46.595680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:46.596863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709016930c9e4fa1842675ff08402a2f', 'request_id': 'req-bbe67f45-6631-4158-be04-584e82032136'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:46.610247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709016930c9e4fa1842675ff08402a2f', 'request_id': 'req-bbe67f45-6631-4158-be04-584e82032136'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:46.627417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709016930c9e4fa1842675ff08402a2f', 'request_id': 'req-bbe67f45-6631-4158-be04-584e82032136'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:46.636765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8f0ce-fe61-4580-9433-d06d0bd1d96c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.660454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] The token cache is being invalidate because project 709016930c9e4fa1842675ff08402a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:46.660800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:46.662086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:46.671980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 867/1752] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:46 2022] DELETE /identity/v3/projects/709016930c9e4fa1842675ff08402a2f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:46.673096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:46.674694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:46.794030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:46.877085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.884535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.885147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.887094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.894638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.903379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:46.922003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe67f45-6631-4158-be04-584e82032136 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:46.957381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:46.958115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:46.958404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:46.958772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:46.959633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:46.961479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:47.000914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 868/1753] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:47.001985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:47.006347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:47.034596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61795-8fa7-4c2d-99e0-0f36580927ac admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:47.044703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 886/1754] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:46 2022] POST /identity/v3/projects => generated 403 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:47.046706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:47.130500 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:47.193707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:47.194673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:47.195153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:47.195591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] PATH_INFO: `/v3/users/ab9d49df938348519755ad17ec33a78c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:47.197234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab9d49df938348519755ad17ec33a78c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:47.199489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:47.344552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab9d49df938348519755ad17ec33a78c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:47.372888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab9d49df938348519755ad17ec33a78c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:47.480351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab9d49df938348519755ad17ec33a78c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:47.485172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab9d49df938348519755ad17ec33a78c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:47.490100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab9d49df938348519755ad17ec33a78c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:47.504477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c14e9-3228-4547-8658-5cba3d236491 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:47.545888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 869/1755] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:47 2022] DELETE /identity/v3/users/ab9d49df938348519755ad17ec33a78c => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:47.549940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:47.554325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:47.560712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:47.582275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:47.582868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:47.583349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:47.589765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:47.597446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:47.711346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 887/1756] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 665 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:47.716611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:47.728696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:47.839270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a5d44-b15b-475c-b961-4e9fbcdde663 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:47.966218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:48.077392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbff28-c675-4b2f-9a80-55ec084e936f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:48.111736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:48.118895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.122198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.122757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.128494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:48.144250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c02c31-cd68-4a91-ad44-7f31a6608ed7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:48.212092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 870/1757] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:47 2022] POST /identity/v3/users => generated 308 bytes in 665 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:48.214086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84530-fc21-440a-90ff-298ca5847629 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.214687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84530-fc21-440a-90ff-298ca5847629 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.215268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84530-fc21-440a-90ff-298ca5847629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.232499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:48.238252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.240022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.242325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] PATH_INFO: `/v3/projects/712f733daf2640129105545e83b905d4/users/1dacf20a43204b07a72d552866ccf786/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.254444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cbe84530-fc21-440a-90ff-298ca5847629 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:48.277845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cbe84530-fc21-440a-90ff-298ca5847629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:48.311867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712f733daf2640129105545e83b905d4, user_id=1dacf20a43204b07a72d552866ccf786, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:48.328198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:48.365068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 888/1758] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:47 2022] PUT /identity/v3/projects/712f733daf2640129105545e83b905d4/users/1dacf20a43204b07a72d552866ccf786/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:48.366167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:48.454196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 871/1759] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:48.455765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.456069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.460917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.526425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] MFA Rules not processed for user `1dacf20a43204b07a72d552866ccf786`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:48.530356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:48.577017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fac29-d599-4f01-8f08-51ff2f653691 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:48.594790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:48.596681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.597289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.597786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.598819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:48.600998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:48.627700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 889/1760] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:48.628986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:48.675350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e3cef-bbc0-4d3f-b201-82864b9dba62 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:48.699997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 872/1761] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:48.701923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:48.706688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:48.715935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731b8f9-c2b4-4457-9a22-f1cc32f1655a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:48.717766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.719706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.723599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] PATH_INFO: `/v3/projects/44a3505e77654afbbf8abf6e7491b4d3/users/1d126faf645141a4a89ce6244a0cc0ee/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.724141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:48.725041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:48.725384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:48.725697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:48.726482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:48.728812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:48.780363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 873/1762] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:48 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:48.784188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:48.818267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a3505e77654afbbf8abf6e7491b4d3, user_id=1d126faf645141a4a89ce6244a0cc0ee, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:48.840884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:48.903358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 890/1763] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:48 2022] PUT /identity/v3/projects/44a3505e77654afbbf8abf6e7491b4d3/users/1d126faf645141a4a89ce6244a0cc0ee/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:48.911155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:48.925517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:48.937458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac251e-5875-4b17-822e-4e4cfa09e2e2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:48.938681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:48.992753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:49.113090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.113926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.114262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.114594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.115467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.118537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.217413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 874/1764] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:48 2022] POST /identity/v3/users => generated 290 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:49.218478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:49.234891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5196ec4-13b3-4b99-8363-270b295f6a05 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.243189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.243975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.244269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.244596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.245860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.247607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.264195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 875/1765] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:49.265308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:49.283404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bd366-f7b5-4089-9bb6-0e21a99c5884 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.299356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.300137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.300507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.300840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.301706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.305946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.334003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 876/1766] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:49.335198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:49.367983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df595da3-bdb0-49b6-a5c2-3e0df7f46399 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.380745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.382092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.382377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.382659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] PATH_INFO: `/v3/projects/44a3505e77654afbbf8abf6e7491b4d3/users/1d126faf645141a4a89ce6244a0cc0ee/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.390071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a3505e77654afbbf8abf6e7491b4d3, user_id=1d126faf645141a4a89ce6244a0cc0ee, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.394514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec155b-f152-475b-8c9c-5b8d716fd117 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.397866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.412262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.413898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.414483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.415151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.426597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 877/1767] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:49 2022] PUT /identity/v3/projects/44a3505e77654afbbf8abf6e7491b4d3/users/1d126faf645141a4a89ce6244a0cc0ee/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:49.437323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:49.624809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35262d9-4284-4fcd-bdcb-403693abb33b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.636554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.637245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.637528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.637806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.638579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.640282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.681045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 878/1768] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:49.686547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:49.710483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.711278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2de8-e4b5-40b6-b0da-ff10724ef4be admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:49.728175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:49.729155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.729441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.729723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] PATH_INFO: `/v3/projects/a11ec63bd8414fe4ae4a7fabc296d28c/users/840f934773474732a7f1804941ae1e8b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.739329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea637e5-b153-45d3-a120-a68e9ec7145a service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.765626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11ec63bd8414fe4ae4a7fabc296d28c, user_id=840f934773474732a7f1804941ae1e8b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:49.768631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:49.812644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 891/1770] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:26:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 910 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:26:49.822595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:49.823131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:49.823549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:49.833991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 879/1770] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:49 2022] PUT /identity/v3/projects/a11ec63bd8414fe4ae4a7fabc296d28c/users/840f934773474732a7f1804941ae1e8b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:26:50.033427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] MFA Rules not processed for user `840f934773474732a7f1804941ae1e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:50.036936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:50.233536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:50.234282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 892/1771] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:50.308028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:50.388074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f69b-75ab-4c49-a4c4-a9eaa361bfb3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:50.410029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:50.410029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:50.410029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:50.410029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:50.410969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:50.410969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:50.470826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:50.477192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 880/1772] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:50 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:26:50.491093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2ce3c-77ed-4d0d-af2b-7c04c8775906 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:50.589286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:50.672166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:50.672166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:50.672623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:50.673025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:50.673846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:50.676225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:50.730368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:50.731049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 893/1773] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:50 2022] POST /identity/v3/users => generated 295 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:50.746501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:50.750162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b5e4d-6f6f-430d-ac00-dadca0404c0b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:50.751923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:50.776668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:50.777470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:50.777825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:50.778182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:50.779143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:50.788490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:50.791405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fab134-1e51-4005-8ddb-1d8962cdac58 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:50.808250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:50.811458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:50.811864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:50.819351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] PATH_INFO: `/v3/projects/0499ee310cb34f02a2307b453710a571` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:50.821169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0499ee310cb34f02a2307b453710a571)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:50.824542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:50.851612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 881/1774] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:50.852757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:50.901417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b30d7-d9f9-4129-a3cb-0340d6d5bc7c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:50.920186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:50.921154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:50.921430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:50.921734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] PATH_INFO: `/v3/projects/475670cf8a3b4c6c8691759f7af66624/users/22f0e172f06a438788d56049c46c5c16/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:50.984038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475670cf8a3b4c6c8691759f7af66624, user_id=22f0e172f06a438788d56049c46c5c16, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:50.987227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:51.042971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0499ee310cb34f02a2307b453710a571', 'request_id': 'req-2fd41608-8643-45d2-8a49-daf3e5e90a66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:51.061003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 882/1775] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:50 2022] PUT /identity/v3/projects/475670cf8a3b4c6c8691759f7af66624/users/22f0e172f06a438788d56049c46c5c16/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:51.062762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:51.084294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0499ee310cb34f02a2307b453710a571', 'request_id': 'req-2fd41608-8643-45d2-8a49-daf3e5e90a66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:51.117645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0499ee310cb34f02a2307b453710a571', 'request_id': 'req-2fd41608-8643-45d2-8a49-daf3e5e90a66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:51.157018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] The token cache is being invalidate because project 0499ee310cb34f02a2307b453710a571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:26:51.160248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:51.169491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2fd41608-8643-45d2-8a49-daf3e5e90a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:51.178414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 894/1776] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:26:50 2022] DELETE /identity/v3/projects/0499ee310cb34f02a2307b453710a571 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:51.184122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d3f0-9d78-4001-8557-b028838d5c9b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.185874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d3f0-9d78-4001-8557-b028838d5c9b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.186634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7d3f0-9d78-4001-8557-b028838d5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.230975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-42d7d3f0-9d78-4001-8557-b028838d5c9b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:51.234966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-42d7d3f0-9d78-4001-8557-b028838d5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:51.304746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ddf5-d12f-4aa6-8a2a-5875350bef56 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:51.319683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:51.320462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.321229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.321624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.322581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:51.326831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:51.359004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 883/1777] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:51.359914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:51.362870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:51.406961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008b2b0-96de-4c7a-b49e-bc1aad2a922b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:51.429786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:51.430879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.431100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.431363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] PATH_INFO: `/v3/projects/475670cf8a3b4c6c8691759f7af66624/users/22f0e172f06a438788d56049c46c5c16/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.446980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475670cf8a3b4c6c8691759f7af66624, user_id=22f0e172f06a438788d56049c46c5c16, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:51.447755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 895/1778] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:51.457587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.457895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.458119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.462356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95e385-b98c-4987-a810-7b1910462da4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:51.497311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:51.498732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:51.503742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 884/1779] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:51 2022] PUT /identity/v3/projects/475670cf8a3b4c6c8691759f7af66624/users/22f0e172f06a438788d56049c46c5c16/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:51.515751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.516043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.516610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.612750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] MFA Rules not processed for user `22f0e172f06a438788d56049c46c5c16`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:51.614107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:51.648960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:51.704574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 896/1780] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:26:51.709571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:51.735271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f58b2-a4e2-4403-9b79-2e44eb042da5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:51.750885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:51.751641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.752101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.756297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.757181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:51.757424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:26:51.757632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:51.763398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:51.807708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 897/1781] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:26:51.816668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:51.819582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 885/1782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:51.826513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:51.828062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:51.874024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67322eb5-a72d-4a73-9f99-3e390b3cd6e8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:51.899858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:51.900614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:51.900918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:51.901175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:51.901906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:51.915542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.009787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb1e18-7a4d-49d1-a9a0-297b13bf25e6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.018855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.032516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.032828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.033086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.039185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 898/1783] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:51 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:52.044246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:52.077655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:52.090543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.141125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 886/1784] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:52.226966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:52.240068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d353b-e765-4943-8760-f1e132645acd service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.402036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591019a-e02e-40f9-8965-33a54aa150ae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.431262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.443824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.445776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.446702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.448217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:52.451955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.454487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.454893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.455675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.457110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.462404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:52.545187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 899/1785] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:52 2022] POST /identity/v3/users => generated 289 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:52.546956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:52.605465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8846275-b1ff-4907-905c-548afbfe2111 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.631426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.631426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.631426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.632439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.634212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:52.637171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.684296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:52.689555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae307c6c-afe7-4803-afbc-12da915261d2 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.713527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 900/1786] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:52.715819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:52.753744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 887/1787] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:26:52.758968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.763237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.763761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.802810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe8f4c-29ca-4375-8163-f223d04d3046 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.814516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.827519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.828201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.829452 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:52.834204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:52.839794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:52.875209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 901/1788] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:52.887172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:52.930973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca3e1e-47a1-4d2b-8ffd-95a4284eeb7d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:52.943992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:52.945316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:52.955727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:52.957118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:52.957118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:52.957118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] PATH_INFO: `/v3/projects/98b0be4e60804ee7a939d2a823b97f97/users/ba9093b12d7744f583523a0dded6ecf8/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:53.021521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b0be4e60804ee7a939d2a823b97f97, user_id=ba9093b12d7744f583523a0dded6ecf8, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:53.030460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab81010-fd1d-4801-9f84-5eedbefa1d18 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:53.096686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 902/1789] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:52 2022] PUT /identity/v3/projects/98b0be4e60804ee7a939d2a823b97f97/users/ba9093b12d7744f583523a0dded6ecf8/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:53.098273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:53.101032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:53.103436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:53.189134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] MFA Rules not processed for user `ba9093b12d7744f583523a0dded6ecf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:53.198774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:53.238259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:53.313587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 888/1790] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jun 27 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 559 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:53.381235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:53.385000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:53.429871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:53.486155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bb325-48da-47de-9ccb-03932de2e027 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:53.524746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 903/1791] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:53.528694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:53.749413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4702f-51e8-44f9-a56e-d5eabf2329d0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:53.761599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:53.764051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:53.764882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:53.766132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:53.768407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:53.782353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:53.794346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:53.795382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:53.796101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:53.796616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:53.833026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:53.835023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:53.862585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 889/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:26:53.869931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 904/1793] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:26:53 2022] POST /identity/v3/projects => generated 381 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:53.871187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:53.889242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41544848-b132-441b-a2ab-0136c45436a5 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:54.138332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:54.139380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:54.143032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:54.144797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:54.146750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:54.154926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:54.263303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 890/1794] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:26:53 2022] POST /identity/v3/users => generated 297 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:26:54.269295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:54.343326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827b92-2042-4d2b-a0b4-ca30710e442b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:54.422588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:54.433241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:54.433680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:54.434323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:54.451530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:54.464469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:54.530533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 905/1795] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:54.530707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:54.547467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491b614-d293-4726-bfed-7553732b3fd5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:54.562092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:54.563030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:54.563310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:54.563583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] PATH_INFO: `/v3/projects/2958d4357510478bbc1bcb52b406e546/users/e1b1207d56df4e8588f40b39cdbe8bb1/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:54.646608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2958d4357510478bbc1bcb52b406e546, user_id=e1b1207d56df4e8588f40b39cdbe8bb1, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:54.649571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:54.712917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 891/1796] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:54 2022] PUT /identity/v3/projects/2958d4357510478bbc1bcb52b406e546/users/e1b1207d56df4e8588f40b39cdbe8bb1/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:26:54.715827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:54.767787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b289-b1d9-4703-b4d2-e8fe6f44b4d6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:55.114867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:55.115574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:55.115856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:55.116126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:55.116941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:55.121072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767995c3-f966-49cc-a2f2-a543cc236b84 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:55.159390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 906/1797] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:26:55.163946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:55.185089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f953d-3d87-4597-9ba9-691f7a303bbc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:55.200800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:55.201752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:55.202030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:55.202308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] PATH_INFO: `/v3/projects/2958d4357510478bbc1bcb52b406e546/users/e1b1207d56df4e8588f40b39cdbe8bb1/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:55.212678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2958d4357510478bbc1bcb52b406e546, user_id=e1b1207d56df4e8588f40b39cdbe8bb1, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:55.218946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:55.282059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 892/1798] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:26:55 2022] PUT /identity/v3/projects/2958d4357510478bbc1bcb52b406e546/users/e1b1207d56df4e8588f40b39cdbe8bb1/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:26:55.294720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de37d10-30b4-4ad2-b4f8-2fe4aacd0d92 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:55.295625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de37d10-30b4-4ad2-b4f8-2fe4aacd0d92 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:55.296129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de37d10-30b4-4ad2-b4f8-2fe4aacd0d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:55.448733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8de37d10-30b4-4ad2-b4f8-2fe4aacd0d92 None None] MFA Rules not processed for user `e1b1207d56df4e8588f40b39cdbe8bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:55.452104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8de37d10-30b4-4ad2-b4f8-2fe4aacd0d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:55.663376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:55.735709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 907/1799] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:55.808012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:55.811171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:56.084062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:26:56.095116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c44dc-06fa-43ff-a4db-62008da3f5d4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:56.095694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c44dc-06fa-43ff-a4db-62008da3f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:56.095694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c44dc-06fa-43ff-a4db-62008da3f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:56.114637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a612c5-9fd2-48c0-bee0-78fa4e392ac2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:56.125503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-660c44dc-06fa-43ff-a4db-62008da3f5d4 None None] MFA Rules not processed for user `995fa1760f5241588a493e5f894b0e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:56.126756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-660c44dc-06fa-43ff-a4db-62008da3f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:56.155400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:56.160714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:56.161610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:56.162652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:56.217886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:56.222161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:56.293111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 893/1800] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:56.347458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 908/1801] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:56.452052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:56.482879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d2154-211f-487d-8ca5-19e73bb45b9d service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:56.502122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:56.504012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:56.504561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:56.505140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:56.771769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:56.774541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:56.852854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 894/1802] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:26:58.559000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fa7fd-1831-450a-a6b9-74a01b43e25d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:58.560234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fa7fd-1831-450a-a6b9-74a01b43e25d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:58.560413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fa7fd-1831-450a-a6b9-74a01b43e25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:58.653893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2e5fa7fd-1831-450a-a6b9-74a01b43e25d None None] MFA Rules not processed for user `e9dab30db0a44655828384db53df29f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:26:58.656035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e5fa7fd-1831-450a-a6b9-74a01b43e25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:26:58.983378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 909/1803] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:26:59.302943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:26:59.654359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bf8ca-8d4c-44b2-819b-7f858c5b5990 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:26:59.677576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:26:59.679428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:26:59.680414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:26:59.681214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] PATH_INFO: `/v3/users/1dacf20a43204b07a72d552866ccf786` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:26:59.690190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dacf20a43204b07a72d552866ccf786)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:26:59.700739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:26:59.901302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dacf20a43204b07a72d552866ccf786'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:59.936650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dacf20a43204b07a72d552866ccf786'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:59.984568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dacf20a43204b07a72d552866ccf786'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:59.988352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dacf20a43204b07a72d552866ccf786'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:26:59.993663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dacf20a43204b07a72d552866ccf786'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:00.067246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 895/1804] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:26:59 2022] DELETE /identity/v3/users/1dacf20a43204b07a72d552866ccf786 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:00.073385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a15718-c887-49f5-8f32-52ac4d2ea714 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:00.074464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a15718-c887-49f5-8f32-52ac4d2ea714 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:00.074973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a15718-c887-49f5-8f32-52ac4d2ea714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:00.131576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-28a15718-c887-49f5-8f32-52ac4d2ea714 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:00.150745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-28a15718-c887-49f5-8f32-52ac4d2ea714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:00.405148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 910/1805] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:00.527566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:00.530249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:00.601776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:00.688173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bb9b4-fdde-44c1-8fde-7611cc48cdcb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:00.692597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bb9b4-fdde-44c1-8fde-7611cc48cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:00.692597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bb9b4-fdde-44c1-8fde-7611cc48cdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:00.795622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-015bb9b4-fdde-44c1-8fde-7611cc48cdcb None None] MFA Rules not processed for user `1d126faf645141a4a89ce6244a0cc0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:00.798530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-015bb9b4-fdde-44c1-8fde-7611cc48cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:00.874038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:00.973153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 911/1806] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:01.019100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c527e-d142-4fae-8554-309782d08c2b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:01.027410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:01.028446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:01.028863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:01.029258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:01.059869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:01.061497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0df13e-4e86-4e86-92f5-4ac9688ca252 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:01.091530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 896/1807] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:02.510124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2208-c646-4593-9dda-55a6fcef433c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:02.511072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d726d-60f7-4bef-9962-52bed63e97fe None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:02.511755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d726d-60f7-4bef-9962-52bed63e97fe None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:02.511755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d726d-60f7-4bef-9962-52bed63e97fe None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:02.513990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 897/1808] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:27:02.514300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2208-c646-4593-9dda-55a6fcef433c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:02.514829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d2208-c646-4593-9dda-55a6fcef433c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:02.518149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 912/1809] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:27:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:27:02.655169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:02.655475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed1c6-1a02-4135-99c6-03f7ced10311 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:02.655475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed1c6-1a02-4135-99c6-03f7ced10311 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:02.655475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed1c6-1a02-4135-99c6-03f7ced10311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:02.655883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:02.655883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:02.735664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:02.742917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:02.757753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7b9ed1c6-1a02-4135-99c6-03f7ced10311 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:02.759119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7b9ed1c6-1a02-4135-99c6-03f7ced10311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:03.126478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 898/1810] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:03.169560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 913/1811] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:03.946345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:03.948515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:03.975777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2587b-90c1-4549-aaeb-0343631136c5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:03.987366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:03.988482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:03.988703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:03.989049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] PATH_INFO: `/v3/projects/712f733daf2640129105545e83b905d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:03.990392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712f733daf2640129105545e83b905d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:03.996419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:04.025003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1d79-d967-47d3-b31d-ba4da0452ecc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:04.027760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1d79-d967-47d3-b31d-ba4da0452ecc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:04.028631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f1d79-d967-47d3-b31d-ba4da0452ecc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:04.032834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 914/1812] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:27:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:27:04.063251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:04.064090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:04.064528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:04.115745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:04.125025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.146130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712f733daf2640129105545e83b905d4', 'request_id': 'req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.159871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712f733daf2640129105545e83b905d4', 'request_id': 'req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.180992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712f733daf2640129105545e83b905d4', 'request_id': 'req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.201650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] The token cache is being invalidate because project 712f733daf2640129105545e83b905d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:04.202386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.206108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.212720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 899/1813] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:03 2022] DELETE /identity/v3/projects/712f733daf2640129105545e83b905d4 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:04.218257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:04.220777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.313126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:04.354299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 915/1814] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:27:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:04.355588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:04.359375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.414506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d380e-b5aa-4ad6-ac25-2dbff4cb2cd5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:04.428604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:04.430245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:04.430644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:04.431073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] PATH_INFO: `/v3/users/840f934773474732a7f1804941ae1e8b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:04.432886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=840f934773474732a7f1804941ae1e8b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:04.435988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:04.467414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:04.507489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da18d3-e026-4db5-83d5-c1846a50ac11 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:04.514969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:04.516569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:04.517006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:04.517308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:04.531057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.551364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:04.555893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:04.556044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840f934773474732a7f1804941ae1e8b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.568311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840f934773474732a7f1804941ae1e8b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.587283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840f934773474732a7f1804941ae1e8b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.588489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840f934773474732a7f1804941ae1e8b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.589245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840f934773474732a7f1804941ae1e8b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:04.615648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 916/1815] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:04.621459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 900/1816] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:04 2022] DELETE /identity/v3/users/840f934773474732a7f1804941ae1e8b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:04.695515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:04.697272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.726087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908df9e-ec57-43d1-947f-7b68ea997fe0 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:04.751736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:04.754227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.885869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:04.886977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:04.887368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:04.887815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:04.890478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:04.921550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:04.923706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:04.988140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 917/1817] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:05.001752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510672e-f2ff-4666-b58d-a5cffcd0e0e0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:05.010792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:05.012730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:05.013406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:05.025380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:05.034241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:05.063894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:05.074549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:05.115858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 901/1818] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:06.707382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:06.709929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:07.167318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25394d4b-9546-4106-9b55-7da31f5ccf32 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:07.194473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:07.196260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:07.201569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:07.202375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] PATH_INFO: `/v3/projects/a11ec63bd8414fe4ae4a7fabc296d28c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:07.208184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11ec63bd8414fe4ae4a7fabc296d28c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:07.247167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:07.476810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11ec63bd8414fe4ae4a7fabc296d28c', 'request_id': 'req-0f97ff99-2c4e-4874-89f2-c3293060810a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.511733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:07.514908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:07.535256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11ec63bd8414fe4ae4a7fabc296d28c', 'request_id': 'req-0f97ff99-2c4e-4874-89f2-c3293060810a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.568188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f2b2f-75bc-4009-9575-5864b4199dc8 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:07.589442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11ec63bd8414fe4ae4a7fabc296d28c', 'request_id': 'req-0f97ff99-2c4e-4874-89f2-c3293060810a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.602589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:07.650778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] The token cache is being invalidate because project a11ec63bd8414fe4ae4a7fabc296d28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:07.651278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.652960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:07.663407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.670730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 918/1819] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:06 2022] DELETE /identity/v3/projects/a11ec63bd8414fe4ae4a7fabc296d28c => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:07.688291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:07.700547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:07.774303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:07.775542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:07.776161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:07.776743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] PATH_INFO: `/v3/users/68c55f4416bc48768f6d043006da2b1e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:07.782433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c55f4416bc48768f6d043006da2b1e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:07.788487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:07.892967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97ff99-2c4e-4874-89f2-c3293060810a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:07.903204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:07.910213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:07.910845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:07.911010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] PATH_INFO: `/v3/users/22f0e172f06a438788d56049c46c5c16` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:07.913122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f0e172f06a438788d56049c46c5c16)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:07.916444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:07.917682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c55f4416bc48768f6d043006da2b1e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.939404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c55f4416bc48768f6d043006da2b1e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.974900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c55f4416bc48768f6d043006da2b1e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.977116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c55f4416bc48768f6d043006da2b1e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:07.980956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c55f4416bc48768f6d043006da2b1e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.026649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 902/1820] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:07 2022] DELETE /identity/v3/users/68c55f4416bc48768f6d043006da2b1e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:08.028256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f0e172f06a438788d56049c46c5c16'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.031309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f0e172f06a438788d56049c46c5c16'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.032281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f0e172f06a438788d56049c46c5c16'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.045843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f0e172f06a438788d56049c46c5c16'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.073135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f0e172f06a438788d56049c46c5c16'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:08.160409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 919/1821] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:07 2022] DELETE /identity/v3/users/22f0e172f06a438788d56049c46c5c16 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:08.607367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:08.611630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:08.871140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27bdfb-e13a-4167-9a10-537346eb9b01 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:09.078015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:09.106884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:09.113802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:09.114492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:09.114783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] PATH_INFO: `/v3/users/ba9093b12d7744f583523a0dded6ecf8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:09.117297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba9093b12d7744f583523a0dded6ecf8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:09.121030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:09.241527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:09.245658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:09.291839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fef7e-0e4d-446e-a6da-23b1e966e3bd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:09.302485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:09.304227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:09.304963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:09.305663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] PATH_INFO: `/v3/projects/5f9abf50927347828b75d5a8763c35ce` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:09.308802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9abf50927347828b75d5a8763c35ce)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:09.313309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:09.340771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba9093b12d7744f583523a0dded6ecf8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.359280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba9093b12d7744f583523a0dded6ecf8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.432886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba9093b12d7744f583523a0dded6ecf8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.436681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba9093b12d7744f583523a0dded6ecf8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.438361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-86064a31-c922-4931-8566-4f64baa93290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba9093b12d7744f583523a0dded6ecf8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.477822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9abf50927347828b75d5a8763c35ce', 'request_id': 'req-dd232d80-310e-4b5a-8887-5a10616d020b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.482792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 903/1822] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:08 2022] DELETE /identity/v3/users/ba9093b12d7744f583523a0dded6ecf8 => generated 0 bytes in 876 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:09.489179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:09.489791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:09.490433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:09.493736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9abf50927347828b75d5a8763c35ce', 'request_id': 'req-dd232d80-310e-4b5a-8887-5a10616d020b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.510558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9abf50927347828b75d5a8763c35ce', 'request_id': 'req-dd232d80-310e-4b5a-8887-5a10616d020b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.520998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:09.541314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] The token cache is being invalidate because project 5f9abf50927347828b75d5a8763c35ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:09.542353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.544506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:09.553396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 920/1823] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:09 2022] DELETE /identity/v3/projects/5f9abf50927347828b75d5a8763c35ce => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:09.555638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:09.559226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:09.576494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:09.580944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:09.760613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d80-310e-4b5a-8887-5a10616d020b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:09.799014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 904/1824] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:09.903203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:09.904925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:09.905424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:09.905923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] PATH_INFO: `/v3/users/1d126faf645141a4a89ce6244a0cc0ee` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:09.908285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d126faf645141a4a89ce6244a0cc0ee)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:09.912187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:09.929079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:09.933053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:10.048813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d126faf645141a4a89ce6244a0cc0ee'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.050593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d126faf645141a4a89ce6244a0cc0ee'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.052997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d126faf645141a4a89ce6244a0cc0ee'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.075256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d126faf645141a4a89ce6244a0cc0ee'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.094773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d126faf645141a4a89ce6244a0cc0ee'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.143400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab2059-6ed9-4e95-bb8f-093c7748cc43 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:10.148766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 921/1825] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:09 2022] DELETE /identity/v3/users/1d126faf645141a4a89ce6244a0cc0ee => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:10.236457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:10.305085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:10.307389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:10.308290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:10.309055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:10.314720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:10.460376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:10.460376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4df42-3ca2-487a-a68c-8b299812f995 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:10.469525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 905/1826] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:10.625496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:10.628793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:10.670129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9b8dc-cdab-43e0-83e3-aa140bd4f02e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:10.693701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:10.697002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:10.697430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:10.698063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] PATH_INFO: `/v3/projects/475670cf8a3b4c6c8691759f7af66624` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:10.701139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=475670cf8a3b4c6c8691759f7af66624)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:10.708937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:10.751008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:10.856894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475670cf8a3b4c6c8691759f7af66624', 'request_id': 'req-579ae562-a195-49f1-a824-42484faf95c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.867387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475670cf8a3b4c6c8691759f7af66624', 'request_id': 'req-579ae562-a195-49f1-a824-42484faf95c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.876796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475670cf8a3b4c6c8691759f7af66624', 'request_id': 'req-579ae562-a195-49f1-a824-42484faf95c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.896490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] The token cache is being invalidate because project 475670cf8a3b4c6c8691759f7af66624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:10.897232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.906916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:10.909448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 922/1827] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:10 2022] DELETE /identity/v3/projects/475670cf8a3b4c6c8691759f7af66624 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:10.917395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891dca-cc7a-4f89-910a-7f7b3feae723 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:10.918032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891dca-cc7a-4f89-910a-7f7b3feae723 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:10.918370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891dca-cc7a-4f89-910a-7f7b3feae723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:10.954544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a5891dca-cc7a-4f89-910a-7f7b3feae723 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:10.957168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a5891dca-cc7a-4f89-910a-7f7b3feae723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:10.991058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:10.992920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:11.109374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ae562-a195-49f1-a824-42484faf95c6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.112557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 906/1828] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:11.115849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.117232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.117805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.118198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] PATH_INFO: `/v3/projects/44a3505e77654afbbf8abf6e7491b4d3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.119694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.120509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.120683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a3505e77654afbbf8abf6e7491b4d3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.121566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.123609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.160138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:11.161726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:11.180194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 907/1829] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:11.181384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.195433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d536043-0d6b-48f6-8a38-f9066a708503 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.202801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.204129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.204692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.206100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.207454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:11.208154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.211407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.234055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 908/1830] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:11.238496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.241109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a3505e77654afbbf8abf6e7491b4d3', 'request_id': 'req-d222b23d-bd16-412a-bf79-48cb9f3556fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:11.257233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a3505e77654afbbf8abf6e7491b4d3', 'request_id': 'req-d222b23d-bd16-412a-bf79-48cb9f3556fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:11.291131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a3505e77654afbbf8abf6e7491b4d3', 'request_id': 'req-d222b23d-bd16-412a-bf79-48cb9f3556fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:11.296613 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b6c99-ecaf-457f-a61c-8bf4f7811e79 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.333339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] The token cache is being invalidate because project 44a3505e77654afbbf8abf6e7491b4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:11.333897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:11.335826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d222b23d-bd16-412a-bf79-48cb9f3556fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:11.344469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 923/1831] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:10 2022] DELETE /identity/v3/projects/44a3505e77654afbbf8abf6e7491b4d3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:11.347435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef366-6205-4c93-b162-2115abbccb4f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.347940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef366-6205-4c93-b162-2115abbccb4f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.348844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef366-6205-4c93-b162-2115abbccb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.395741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a71ef366-6205-4c93-b162-2115abbccb4f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:11.398922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a71ef366-6205-4c93-b162-2115abbccb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:11.422214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.423235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.423531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.423814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.424887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.427248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.435709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:11.477034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 909/1832] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:11.478546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.482588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:11.486007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:11.570902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 924/1833] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:11.580588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.580904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.581161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.583610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a5b-d802-467a-95ed-c14036c62f54 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.589659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.590518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.590892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.592179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.593152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.595608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.605418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:11.607492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:11.622642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 925/1834] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:11.627293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 910/1835] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:11.632574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.634159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.648014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895a9a-ba30-4462-9c9c-311d9ccd19bd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.649501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921c223-3af0-4adf-9fd2-e23b8f6c9cfa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.656234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.657702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.658633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.659330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.659472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.660214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.660738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.662130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:11.663203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.666491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.671283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.675963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.681580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.691595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 926/1836] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:11.696646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.713296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 911/1837] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:11.718250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.721313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885a41-4261-4e7a-9f38-d3abdea88ab4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.729894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.731180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.731778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.732430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.733811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.738346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.744659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeee69-f2b8-4dec-b75c-67bea921e71a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.752451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.753579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.754165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.754468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.755353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.757838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.783405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 912/1838] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:11.790699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.807335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a937142-9ffe-4444-93d7-6f4f614a2498 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:11.814053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:11.815457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:11.816021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:11.816573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] PATH_INFO: `/v3/projects/c1bff836d1da41c9b5bb85edd595cd4a/users/ff8c3de0c861494393d8517c092062dc/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:11.830465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 927/1839] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:11.836654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.872619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1bff836d1da41c9b5bb85edd595cd4a, user_id=ff8c3de0c861494393d8517c092062dc, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:11.876186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:11.914544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 913/1840] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:11 2022] PUT /identity/v3/projects/c1bff836d1da41c9b5bb85edd595cd4a/users/ff8c3de0c861494393d8517c092062dc/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:11.915915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:11.937293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.099974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a7508-a843-4dc0-9a59-bcdb7f1d455f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.106933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.107353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.107637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.107918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] PATH_INFO: `/v3/projects/98b0be4e60804ee7a939d2a823b97f97` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.109516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b0be4e60804ee7a939d2a823b97f97)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.112078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:12.162863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:12.183056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b0be4e60804ee7a939d2a823b97f97', 'request_id': 'req-95674236-1c71-4255-9290-82f708cc4bcb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.196386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed7aa5-62fd-4903-b52c-e3c594932ae5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.197135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b0be4e60804ee7a939d2a823b97f97', 'request_id': 'req-95674236-1c71-4255-9290-82f708cc4bcb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.216099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b0be4e60804ee7a939d2a823b97f97', 'request_id': 'req-95674236-1c71-4255-9290-82f708cc4bcb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.239329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] The token cache is being invalidate because project 98b0be4e60804ee7a939d2a823b97f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:12.241069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.242255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-95674236-1c71-4255-9290-82f708cc4bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.247901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 914/1841] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:11 2022] DELETE /identity/v3/projects/98b0be4e60804ee7a939d2a823b97f97 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:12.256264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.257588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.258094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.317242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] MFA Rules not processed for user `ff8c3de0c861494393d8517c092062dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:12.321459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.412053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.413161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.413613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.414172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.415797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.418269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:12.460750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 928/1842] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:11 2022] POST /identity/v3/users => generated 283 bytes in 625 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:12.461911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:12.466094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.482029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af6dd8-6636-4815-b514-8828e06f1a14 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.483219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 915/1843] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:12.484201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:12.490139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.501888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.503842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.504138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.506518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] PATH_INFO: `/v3/users/e1b1207d56df4e8588f40b39cdbe8bb1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.507644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47703-1b54-45d5-b75f-d180590f4844 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.510706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b1207d56df4e8588f40b39cdbe8bb1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.512849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:12.526191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.527229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.534121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.534717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.535862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.537816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:12.564154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 916/1844] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:12.565629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:12.571206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.650045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:12.723052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b1207d56df4e8588f40b39cdbe8bb1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.725588 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b1207d56df4e8588f40b39cdbe8bb1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.727093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b1207d56df4e8588f40b39cdbe8bb1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.746152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b1207d56df4e8588f40b39cdbe8bb1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.761759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b1207d56df4e8588f40b39cdbe8bb1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:12.768044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54edbee-c4db-4c81-a424-e0ee393e4ed0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.779159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.780599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.780599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.780599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.789285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.810234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 929/1845] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:12 2022] DELETE /identity/v3/users/e1b1207d56df4e8588f40b39cdbe8bb1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:12.810891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:12.816664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:12.818875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.820451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:12.847373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 917/1846] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:12.947952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1535ed-4428-4471-8830-ed4a0190d046 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:12.961089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:12.963613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:12.964207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:12.964921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:12.966721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:12.971709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa356399-1005-44bd-bbd5-ed37f7abd52c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:13.042972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 930/1847] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:13.047333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:13.063716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b2b04-6133-479a-bfee-a68622ceed6f service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:13.077777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:13.078955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:13.081666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:13.082420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] PATH_INFO: `/v3/projects/45ce9d44878c4c53a87b18e93af9a929/users/1c61fa258b4440c8888ced21cfc63d78/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:13.128885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ce9d44878c4c53a87b18e93af9a929, user_id=1c61fa258b4440c8888ced21cfc63d78, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:13.137024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:13.171336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 918/1848] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:13 2022] PUT /identity/v3/projects/45ce9d44878c4c53a87b18e93af9a929/users/1c61fa258b4440c8888ced21cfc63d78/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:13.182132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:13.182474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:13.182702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:13.226235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] MFA Rules not processed for user `1c61fa258b4440c8888ced21cfc63d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:13.227595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:13.429389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 931/1849] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:13.436908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:13.488897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:13.601997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:13.604382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:13.641527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff8cb3-d500-46f5-9c79-83eead0f13e8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:13.662777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:13.663928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:13.664435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:13.664921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:13.666157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:13.669938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:13.732201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:13.802668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 919/1850] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:13 2022] POST /identity/v3/projects => generated 365 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:13.810218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:13.881874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a628a-d979-40f5-80bd-196bf9e91cea None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.000053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f2be-76c6-464c-84cf-b5eff36aff5f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.005994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.007189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.007754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.008082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.008477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.008685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.008995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.009282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] PATH_INFO: `/v3/projects/2958d4357510478bbc1bcb52b406e546` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.009950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.011953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2958d4357510478bbc1bcb52b406e546)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.012817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.016119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.049594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 920/1851] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:13 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:14.053682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.073553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bc1d3-ef50-4fbb-b037-15dfce3de60e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.082005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.082733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.083183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.084099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.085115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.087868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.100684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 921/1852] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:14.105141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.119907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39150f5b-e8a6-4a39-a17c-3101d73d041e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.120788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2958d4357510478bbc1bcb52b406e546', 'request_id': 'req-74c88e81-9037-4272-90f9-ce21f8f0ea37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:14.130995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2958d4357510478bbc1bcb52b406e546', 'request_id': 'req-74c88e81-9037-4272-90f9-ce21f8f0ea37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:14.150769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2958d4357510478bbc1bcb52b406e546', 'request_id': 'req-74c88e81-9037-4272-90f9-ce21f8f0ea37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:14.188161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] The token cache is being invalidate because project 2958d4357510478bbc1bcb52b406e546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:14.188825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:14.190268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-74c88e81-9037-4272-90f9-ce21f8f0ea37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:14.195203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 932/1853] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:13 2022] DELETE /identity/v3/projects/2958d4357510478bbc1bcb52b406e546 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:14.204196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690df0c-0d9f-4711-ab26-9dcf4d184318 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.204979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690df0c-0d9f-4711-ab26-9dcf4d184318 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.208186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690df0c-0d9f-4711-ab26-9dcf4d184318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.255423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a690df0c-0d9f-4711-ab26-9dcf4d184318 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:14.258815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a690df0c-0d9f-4711-ab26-9dcf4d184318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:14.266574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.268855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.269593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.272155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] PATH_INFO: `/v3/projects/7223287313ea4ee28915d613dac3221e/users/4ad5fa01ef5949a99b23bd9d9af47e14/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.292241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 933/1854] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:14.295875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.296575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.297182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.317430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7223287313ea4ee28915d613dac3221e, user_id=4ad5fa01ef5949a99b23bd9d9af47e14, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.322856 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.337472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:14.341133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:14.359297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 922/1855] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:14 2022] PUT /identity/v3/projects/7223287313ea4ee28915d613dac3221e/users/4ad5fa01ef5949a99b23bd9d9af47e14/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:14.379387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.384632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:14.402907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:14.498058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:14.539738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 934/1856] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:14.544630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.556747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fcf48-4c40-431f-b2af-b62e2443f333 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.563863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.564521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.565030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.565405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.566189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:14.566498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.570316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.586346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee595-225d-4404-b5b1-9e274d3c03ab admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.588911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 935/1857] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:27:14.590582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.595861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.597459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.598086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.598832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.600593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.603773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.604778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a81b2-3d9c-49da-a203-b39a0b9c40c3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.614927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.617140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.617683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.618159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.619092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.621414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.626703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 923/1858] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:14.631413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.654279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a897f-f8de-4170-8ca8-f53866fefe46 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.661272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 936/1859] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:14 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:14.667926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:14.668154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.669953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.670505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.671018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] PATH_INFO: `/v3/projects/7223287313ea4ee28915d613dac3221e/users/4ad5fa01ef5949a99b23bd9d9af47e14/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.675678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7223287313ea4ee28915d613dac3221e, user_id=4ad5fa01ef5949a99b23bd9d9af47e14, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:14.682457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0dd57-9605-4232-8450-0b2eef8963af admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:14.707652 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 924/1860] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:14 2022] PUT /identity/v3/projects/7223287313ea4ee28915d613dac3221e/users/4ad5fa01ef5949a99b23bd9d9af47e14/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:14.726619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.727444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.728245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:14.748265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c2da-b58f-4b7d-ad3b-c1665ff7b6d5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:14.783697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] MFA Rules not processed for user `4ad5fa01ef5949a99b23bd9d9af47e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:14.787180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:14.987251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:14.996127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:14.997442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:14.999285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.019295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.033749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:15.103374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 937/1861] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:14 2022] POST /identity/v3/users => generated 291 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:15.121762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.181365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808c599-cfc3-4444-a10f-717c32cb3bb7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.196946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 925/1862] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:15.199982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.202030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.202662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.203270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.207045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.210772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:15.220927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.224265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:15.259924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 938/1863] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:15.273667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.308786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3a04e-51f1-4d21-96eb-d2145f5733e1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.347292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.348524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.351606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.352126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.353414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.355855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:15.379795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 939/1864] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:15.385791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.400360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3920d1-5be9-4772-ab62-9500468343ec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.407168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.408812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.410041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.410041 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] PATH_INFO: `/v3/projects/c99fb6a1cb554ba9bde66fa32729cef1/users/c1f0aa6314d142ceb912cc4b8d53731f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.435172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99fb6a1cb554ba9bde66fa32729cef1, user_id=c1f0aa6314d142ceb912cc4b8d53731f, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.442949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:15.446346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1a10c-f0e5-4f6f-a991-02c85d384f9c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.451905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.452527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.453044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.453207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.467298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.469047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0e7bf-4fcb-47e3-a958-4a0c7bf6631d service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:15.482133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 940/1866] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:15 2022] PUT /identity/v3/projects/c99fb6a1cb554ba9bde66fa32729cef1/users/c1f0aa6314d142ceb912cc4b8d53731f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:15.482458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 926/1866] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:15.513864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.514545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.515344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.573564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] MFA Rules not processed for user `c1f0aa6314d142ceb912cc4b8d53731f`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:15.577756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:15.583451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:15.645667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.672513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:15.777732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af27c7-e871-4e7d-990f-f9b1b7110a55 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.786555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.792538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 927/1867] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:15.792798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.792798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.792798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.793964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:15.797808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:15.958038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4355-4ac2-4df4-9672-8a4df4f03fb3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:15.969424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:15.970732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:15.971426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:15.971903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:15.973355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:15.976050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.002701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.004771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.017243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 941/1868] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:16.043690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 928/1869] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:15 2022] POST /identity/v3/projects => generated 381 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:16.051331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:16.099466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51080b-47cd-4d78-b040-26ccdb844d4a service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:16.351064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:16.352751 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.353566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.354023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.354918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.357186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.412536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 942/1870] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:16 2022] POST /identity/v3/users => generated 297 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:16.417142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:16.443569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25c015-8168-440a-bcbb-d78654ff283f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:16.462659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:16.463654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.463952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.464230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.465214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.466987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.517070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 929/1871] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:16.521591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:16.541096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f213b-cf40-4b47-a99d-eee7ef3c0519 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:16.557708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:16.558688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.558962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.559241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] PATH_INFO: `/v3/projects/963cdfe3876b40c48e65bc0523ae0deb/users/a62b361267444f90866354426317bddb/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.595293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=963cdfe3876b40c48e65bc0523ae0deb, user_id=a62b361267444f90866354426317bddb, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.608592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.634944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 943/1872] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:16 2022] PUT /identity/v3/projects/963cdfe3876b40c48e65bc0523ae0deb/users/a62b361267444f90866354426317bddb/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:16.639665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:16.679699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b49af5-5545-4827-8537-f1e65eb854d2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:16.821295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:16.822204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.822492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.822789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.823796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.825504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c7cad-a4d3-427b-8810-0227af9d7ade admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.852710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 930/1873] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:16.854934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:16.867230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df8f72-3573-48fe-935b-bd379aafb50c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:16.874164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:16.875746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.875746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.875746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] PATH_INFO: `/v3/projects/963cdfe3876b40c48e65bc0523ae0deb/users/a62b361267444f90866354426317bddb/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.879174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963cdfe3876b40c48e65bc0523ae0deb, user_id=a62b361267444f90866354426317bddb, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:16.882165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:16.897739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 944/1874] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:16 2022] PUT /identity/v3/projects/963cdfe3876b40c48e65bc0523ae0deb/users/a62b361267444f90866354426317bddb/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:16.909839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:16.910595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:16.911109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:16.974274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] MFA Rules not processed for user `a62b361267444f90866354426317bddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:16.980627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:16.991486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:17.146185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 931/1875] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:17.167927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:17.170736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:17.214201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:17.276842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfa8a5-0387-47ee-8e10-edcace23adb8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:17.283104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:17.284694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:17.285076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:17.285438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:17.297059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:17.298676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:17.311913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 945/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:17.811986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:17.828710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832a0d8-abd1-44fa-8b17-ef49b17fc000 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:17.836200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62b361267444f90866354426317bddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '963cdfe3876b40c48e65bc0523ae0deb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:17.837498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:17.837993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:17.838479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:17.839631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:17.840103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:17.856146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:17.891090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 932/1877] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:17.898053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:17.915950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd616e0f-466f-479f-b48a-81b8a70b12da service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:17.936892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62b361267444f90866354426317bddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '963cdfe3876b40c48e65bc0523ae0deb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:17.937800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:17.938256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:17.938701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:17.939973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:17.943540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:18.010546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 946/1878] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:17 2022] POST /identity/v3/projects => generated 353 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:18.011950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:18.046248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451902e-0c9b-4952-a7b5-4f6fde30393d tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:18.229486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62b361267444f90866354426317bddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '963cdfe3876b40c48e65bc0523ae0deb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:18.230557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:18.231001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:18.231444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:18.232596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:18.233066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:18.235004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:18.278355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 933/1879] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:27:18.352067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:18.376586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95bd2d-22fa-4f77-bcc0-d5974ac801b2 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:18.398527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a62b361267444f90866354426317bddb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '963cdfe3876b40c48e65bc0523ae0deb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:18.400449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:18.401015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:18.402206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] PATH_INFO: `/v3/projects/02a88e62339e4862b8c118a633ba121b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:18.404096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorizing `identity:delete_project(project_id=02a88e62339e4862b8c118a633ba121b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:18.406466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:18.486211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a88e62339e4862b8c118a633ba121b', 'request_id': 'req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:18.501550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a88e62339e4862b8c118a633ba121b', 'request_id': 'req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:18.514710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a88e62339e4862b8c118a633ba121b', 'request_id': 'req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:18.554926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] The token cache is being invalidate because project 02a88e62339e4862b8c118a633ba121b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:18.555565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:18.557007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f9ea0f1-7244-49d7-818e-2dd8bd5acf50 tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:18.561344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 947/1880] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:18 2022] DELETE /identity/v3/projects/02a88e62339e4862b8c118a633ba121b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:19.103859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eddb7-82e6-4894-8f80-eafba98d7109 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:19.104966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:19.106769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eddb7-82e6-4894-8f80-eafba98d7109 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:19.109133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0eddb7-82e6-4894-8f80-eafba98d7109 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:19.113007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 948/1881] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:27:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:27:19.119812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:19.127388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:19.127388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:19.127388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:19.239928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:19.247471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:19.424632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 949/1882] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:19.464780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:19.467158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:19.499217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df289ab-32cc-40e5-9bfa-a1670886d60a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:19.507440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:19.508559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:19.508976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:19.509356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:19.520930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:19.522846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:19.550563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 950/1883] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:19.551642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:19.555924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:19.579468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:19.589817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb95b19-64af-4df7-acd4-ed396fe7853f tempest-VolumeQuotaClassesTest-1503624934 tempest-VolumeQuotaClassesTest-1503624934-project-admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:19.765007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:19.871253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:19.872014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:19.872300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:19.872615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] PATH_INFO: `/v3/users/c1f0aa6314d142ceb912cc4b8d53731f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:19.874824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f0aa6314d142ceb912cc4b8d53731f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:19.879087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:19.934813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531479-6c13-4caa-87cd-e165a230c6c0 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:19.960878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:19.960878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:19.960878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:19.960878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] PATH_INFO: `/v3/users/1c61fa258b4440c8888ced21cfc63d78` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:19.966892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c61fa258b4440c8888ced21cfc63d78)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:19.985401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:20.007743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f0aa6314d142ceb912cc4b8d53731f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.017358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f0aa6314d142ceb912cc4b8d53731f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.033064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f0aa6314d142ceb912cc4b8d53731f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.034287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f0aa6314d142ceb912cc4b8d53731f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.035029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3617f15e-d64b-4bf0-84f3-327e379aa758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f0aa6314d142ceb912cc4b8d53731f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.066554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 934/1884] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:19 2022] DELETE /identity/v3/users/c1f0aa6314d142ceb912cc4b8d53731f => generated 0 bytes in 960 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:20.069666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:20.070252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:20.074212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:20.095195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c61fa258b4440c8888ced21cfc63d78'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.096343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c61fa258b4440c8888ced21cfc63d78'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.097428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c61fa258b4440c8888ced21cfc63d78'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.111447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c61fa258b4440c8888ced21cfc63d78'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.124953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c7e4c15-c131-4e34-bb2b-c35136f1bf78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c61fa258b4440c8888ced21cfc63d78'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:20.138207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:20.140564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:20.168237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 951/1885] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:19 2022] DELETE /identity/v3/users/1c61fa258b4440c8888ced21cfc63d78 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:20.173209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:20.173478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:20.173815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:20.199618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:20.200974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:20.278944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 935/1886] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:20.339138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:20.340842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:20.476822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:20.499304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bbe8d-4fbf-4627-b223-bfd759929a76 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:20.528092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 952/1887] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:20.581308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:20.582517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:20.582909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:20.583361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:20.613180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:20.614954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:20.660649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 936/1888] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:20.680413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:20.680413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:20.703720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68d5d6-1337-42c3-a816-b80b68c63704 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:20.721441 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:20.724024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:20.724918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:20.726058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:20.754312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:20.764643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:20.803348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 953/1889] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:21.829049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:21.832420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:21.902571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c0ae-e5c4-4b36-b7d5-9ada87011082 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:21.934870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:21.945160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:21.945500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:21.946105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] PATH_INFO: `/v3/projects/c99fb6a1cb554ba9bde66fa32729cef1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:21.967950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99fb6a1cb554ba9bde66fa32729cef1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:21.976240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:22.135575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99fb6a1cb554ba9bde66fa32729cef1', 'request_id': 'req-61973114-113e-4d99-ade2-ae0a49715644'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.162678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99fb6a1cb554ba9bde66fa32729cef1', 'request_id': 'req-61973114-113e-4d99-ade2-ae0a49715644'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.201409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99fb6a1cb554ba9bde66fa32729cef1', 'request_id': 'req-61973114-113e-4d99-ade2-ae0a49715644'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.282301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] The token cache is being invalidate because project c99fb6a1cb554ba9bde66fa32729cef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:22.282666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.292151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.305972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 937/1890] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:21 2022] DELETE /identity/v3/projects/c99fb6a1cb554ba9bde66fa32729cef1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:22.331122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:22.355448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebead0a0-a7a4-4dda-8829-a5cbc6a42c6a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:22.384801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:22.465982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:22.468098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:22.506891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:22.508404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:22.509281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:22.509811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] PATH_INFO: `/v3/users/a62b361267444f90866354426317bddb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:22.511773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62b361267444f90866354426317bddb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:22.515610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:22.611687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62b361267444f90866354426317bddb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.613469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62b361267444f90866354426317bddb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.615088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62b361267444f90866354426317bddb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.620420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973114-113e-4d99-ade2-ae0a49715644 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:22.629775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62b361267444f90866354426317bddb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.630716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:22.632085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:22.632085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:22.632755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] PATH_INFO: `/v3/projects/45ce9d44878c4c53a87b18e93af9a929` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:22.634172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ce9d44878c4c53a87b18e93af9a929)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:22.636761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:22.670365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62b361267444f90866354426317bddb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.702261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:22.712554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 954/1891] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:22 2022] DELETE /identity/v3/users/a62b361267444f90866354426317bddb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:22.726133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ce9d44878c4c53a87b18e93af9a929', 'request_id': 'req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.742982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ce9d44878c4c53a87b18e93af9a929', 'request_id': 'req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.761251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ce9d44878c4c53a87b18e93af9a929', 'request_id': 'req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.786946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] The token cache is being invalidate because project 45ce9d44878c4c53a87b18e93af9a929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:22.787612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.788522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:22.790393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 938/1892] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:22 2022] DELETE /identity/v3/projects/45ce9d44878c4c53a87b18e93af9a929 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:22.795215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:22.797332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:23.009315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef21893-3e69-4c39-a2ea-17b27aca6220 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:23.020517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:23.022195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:23.022715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:23.023185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] PATH_INFO: `/v3/users/4ad5fa01ef5949a99b23bd9d9af47e14` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:23.025859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad5fa01ef5949a99b23bd9d9af47e14)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:23.029448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:23.107391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:23.132572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad5fa01ef5949a99b23bd9d9af47e14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.133612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad5fa01ef5949a99b23bd9d9af47e14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.134867 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad5fa01ef5949a99b23bd9d9af47e14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.150626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad5fa01ef5949a99b23bd9d9af47e14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.169729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5bfc39e1-3454-482e-8708-e8ab6f8d6d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad5fa01ef5949a99b23bd9d9af47e14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.254292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 955/1893] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:22 2022] DELETE /identity/v3/users/4ad5fa01ef5949a99b23bd9d9af47e14 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:23.378748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:23.385798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:23.587395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103a9e0-6cbe-4ab8-b965-0c8fd1072737 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:23.717362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:23.723064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:23.723887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:23.724588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] PATH_INFO: `/v3/projects/963cdfe3876b40c48e65bc0523ae0deb` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:23.729621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963cdfe3876b40c48e65bc0523ae0deb)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:23.748873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:23.887912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963cdfe3876b40c48e65bc0523ae0deb', 'request_id': 'req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.907115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963cdfe3876b40c48e65bc0523ae0deb', 'request_id': 'req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.924419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963cdfe3876b40c48e65bc0523ae0deb', 'request_id': 'req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.935193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:23.949862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] The token cache is being invalidate because project 963cdfe3876b40c48e65bc0523ae0deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:23.950639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.951868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:23.955437 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 939/1894] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:23 2022] DELETE /identity/v3/projects/963cdfe3876b40c48e65bc0523ae0deb => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:23.961274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf3d86-2e8f-4d5b-b75d-76edd071ee42 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:23.961919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf3d86-2e8f-4d5b-b75d-76edd071ee42 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:23.961919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf3d86-2e8f-4d5b-b75d-76edd071ee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.005344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5dcf3d86-2e8f-4d5b-b75d-76edd071ee42 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:24.007651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5dcf3d86-2e8f-4d5b-b75d-76edd071ee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.132693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.135781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.206690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 956/1895] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:24.213494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.214483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.214776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.238758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:24.240225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.248776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 957/1896] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:24.256352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.266445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1fe9d-01fc-4824-b9d0-8da91c4f4375 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.272455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.273245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.273625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.273965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.274778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:24.275422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.276927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.293243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 958/1897] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:27:24.298120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.309664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4daa3-42d4-4a2c-9a4d-ab608bddd7b0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.315686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc7f9a-e549-4166-ad73-e14560dec829 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.319471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.321135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.321606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.322079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] PATH_INFO: `/v3/projects/7223287313ea4ee28915d613dac3221e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.322393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.323193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.323716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.323953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.325704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.326886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7223287313ea4ee28915d613dac3221e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.327086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.332648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.374772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 959/1898] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:24.383727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.445340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7223287313ea4ee28915d613dac3221e', 'request_id': 'req-e334e716-0d7d-4c5a-8b4c-91def0444d2f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:24.466554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7223287313ea4ee28915d613dac3221e', 'request_id': 'req-e334e716-0d7d-4c5a-8b4c-91def0444d2f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:24.494427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7223287313ea4ee28915d613dac3221e', 'request_id': 'req-e334e716-0d7d-4c5a-8b4c-91def0444d2f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:24.534998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] The token cache is being invalidate because project 7223287313ea4ee28915d613dac3221e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:24.536147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:24.538509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e334e716-0d7d-4c5a-8b4c-91def0444d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:24.542016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 940/1899] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:24 2022] DELETE /identity/v3/projects/7223287313ea4ee28915d613dac3221e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:24.554219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ee3c-9769-42b8-afd5-8bb25069eb3e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.556357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ee3c-9769-42b8-afd5-8bb25069eb3e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.558663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582ee3c-9769-42b8-afd5-8bb25069eb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.565639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9892-de28-43ee-95d3-487ff69a2067 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.598147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6582ee3c-9769-42b8-afd5-8bb25069eb3e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:24.601134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6582ee3c-9769-42b8-afd5-8bb25069eb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.627499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:24.770871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 941/1900] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:24.785753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.786394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.788860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.801496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.802319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.802571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.802872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.803636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.814103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.839860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:24.841169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.861662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 942/1901] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:24.866956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 960/1902] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/users => generated 286 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:24.868753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.869371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.872083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:24.884757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed51cd-e93b-42a5-9ab6-baa598cba388 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.896800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.898185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.898824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.899185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecac45-73f9-445b-8072-698f328569dd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:24.899538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.902356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:24.902356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.905697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.916350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:24.917075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:24.917760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:24.918035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:24.918825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:24.920573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:24.953309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 943/1903] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:24.955370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.963801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 961/1904] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:24.974600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:24.992731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe3517-8057-4c16-858e-1157728a1d2d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.007954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.008716 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.009601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.009601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.010210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.012501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.013850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eda4d6-fbd6-41e0-bd43-4d8f9be2dfb9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.022964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:25.032423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.033735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.034432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.034875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.035998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.040111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.094617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 962/1905] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:25.105094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.134473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 944/1906] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:24 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:25.140399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.153046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db994c2-d441-445c-a870-0d6d1b3d92cc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.325594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.327092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.327599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.328132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] PATH_INFO: `/v3/projects/0ca8e18e118641a89e5720920d218293/users/cbcc39771e1d47a68585a8b50187b7ce/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.343677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dbdb8-ab5e-4e14-ac56-1ba31dfe7f63 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.354329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca8e18e118641a89e5720920d218293, user_id=cbcc39771e1d47a68585a8b50187b7ce, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.358227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e77db8-aece-4adf-8200-10f252d2a734 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.365243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.366105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.366398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.366703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.372354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.373154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.409203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 963/1907] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:25 2022] PUT /identity/v3/projects/0ca8e18e118641a89e5720920d218293/users/cbcc39771e1d47a68585a8b50187b7ce/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:25.428556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.429002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.429480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.432293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 945/1908] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:25 2022] POST /identity/v3/users => generated 296 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:25.438088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.468852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef6ffa-826e-4686-bd72-def801f4678e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.499089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] MFA Rules not processed for user `cbcc39771e1d47a68585a8b50187b7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:25.500485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:25.650540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.656503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.657888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.658432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.659557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.662750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.697951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 946/1909] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:25.704531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.720625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f993-f40d-401c-a41f-4b9e42272868 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.740047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.741506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.742071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.742640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.744302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.763410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.784833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:25.824482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 964/1910] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:25.843693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.846130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 947/1911] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:25.853519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:25.857853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:25.884342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045edf8b-1a5a-4b0e-8436-b8ac25b1c116 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:25.899527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:25.900996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:25.900996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:25.901203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] PATH_INFO: `/v3/projects/0a6558beb7204c349dffc9bc625b34f1/users/d84cf10772f04fab94b613aadfaf05ac/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:25.952769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6558beb7204c349dffc9bc625b34f1, user_id=d84cf10772f04fab94b613aadfaf05ac, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:25.960306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41ca7f-3e8c-41aa-b3f2-61418e1b786b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:25.994136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 948/1912] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:25 2022] PUT /identity/v3/projects/0a6558beb7204c349dffc9bc625b34f1/users/d84cf10772f04fab94b613aadfaf05ac/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:26.014199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.014462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.014844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.049076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fd2d3-7682-4e93-ac23-3a418e402387 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.064107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] MFA Rules not processed for user `d84cf10772f04fab94b613aadfaf05ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:26.066937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:26.135791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.137473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.137657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.137904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.139662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:26.198622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 949/1913] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:26.204774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.236475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.240548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.262832 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 965/1914] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:26.287446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.306887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f3fa-21f7-485c-9b7e-6df45656541f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.312055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.313167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.313517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.313957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.315464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.317744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.355293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 950/1915] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:26 2022] POST /identity/v3/projects => generated 391 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:26.360090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.409286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea36134-7965-4e12-9227-d6db86074155 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.425844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:26.470169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26722080-2e36-43de-bda0-e9b98a4b0218 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.477638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.478664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.479146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.479587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.481000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.484551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.525081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 951/1916] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:26 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:26.527646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.529455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.531458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.532145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.532872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] PATH_INFO: `/v3/users/cbcc39771e1d47a68585a8b50187b7ce` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.535468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcc39771e1d47a68585a8b50187b7ce)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.539945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.546879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02f91c-316d-4dae-9a07-9d8a0e1150a1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.554239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.555457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.555936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.556309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.557273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.559673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.578751 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 952/1917] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:26.585475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.601357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ba5c9-b18c-4bc3-bf0f-6b50f7a1269d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.614120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:26.615936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.616859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.617169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] PATH_INFO: `/v3/projects/fa063bb6be7345a2ac6be3c212f6b705/users/1e68cb49f59a46ec98d6d2f5e940c3a9/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.646638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa063bb6be7345a2ac6be3c212f6b705, user_id=1e68cb49f59a46ec98d6d2f5e940c3a9, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:26.649159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcc39771e1d47a68585a8b50187b7ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:26.649993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcc39771e1d47a68585a8b50187b7ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:26.651022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcc39771e1d47a68585a8b50187b7ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:26.653657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:26.667487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcc39771e1d47a68585a8b50187b7ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:26.680795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 953/1918] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:26 2022] PUT /identity/v3/projects/fa063bb6be7345a2ac6be3c212f6b705/users/1e68cb49f59a46ec98d6d2f5e940c3a9/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:26.687012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:26.688110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eb783b31-b097-4096-b105-e1117d41d3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcc39771e1d47a68585a8b50187b7ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:26.690326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:26.708107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:26.745888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 966/1919] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:26 2022] DELETE /identity/v3/users/cbcc39771e1d47a68585a8b50187b7ce => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:26.753088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:26.754559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:26.755084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:26.801918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:26.803900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:26.841889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa52c14-c414-4443-a060-e02da8f724ef admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:26.988621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 967/1920] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:27.005451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:27.007882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:27.054861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:27.057277 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.057762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.058307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.060951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:27.064233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:27.106249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 954/1921] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:27.110103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:27.125911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936322f-a0a9-4600-bb6e-23e069a70d0f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:27.131897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:27.133319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.133747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.134149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] PATH_INFO: `/v3/projects/fa063bb6be7345a2ac6be3c212f6b705/users/1e68cb49f59a46ec98d6d2f5e940c3a9/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.137167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa063bb6be7345a2ac6be3c212f6b705, user_id=1e68cb49f59a46ec98d6d2f5e940c3a9, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:27.139080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:27.140585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6316e41-8abd-4a2c-a800-830a5712c4ca admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:27.168639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 955/1922] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:27 2022] PUT /identity/v3/projects/fa063bb6be7345a2ac6be3c212f6b705/users/1e68cb49f59a46ec98d6d2f5e940c3a9/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:27.185829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.186186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.187469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.222154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8c9b5-aeac-46ee-bed4-a52cfca1a28f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:27.257517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] MFA Rules not processed for user `1e68cb49f59a46ec98d6d2f5e940c3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:27.266764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:27.400355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:27.401111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.401403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.401684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.553713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:27.560274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:27.592922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 968/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:27.608659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 956/1924] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:27.622946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:27.624444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:27.725063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c1439-4001-4b9b-b114-a8d0641bcc4b service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:27.729662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:27.730979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.731247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.731623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.748697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:27.750526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:27.766026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 969/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:27.899388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:27.901070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:27.914568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4501b-eaa6-4801-a208-90727d94f9d4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:27.919413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:27.919735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:27.919892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:27.920147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:27.922323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:27.948860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:28.010623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.012969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be9d9c-678f-4551-890e-2ecb873f67b6 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.030456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 957/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:28.165483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.168187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:28.190981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983d103-e854-431e-98e8-fce119ace44a service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.200592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.202008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.202175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.202570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] PATH_INFO: `/v3/projects/0ca8e18e118641a89e5720920d218293` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.204274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca8e18e118641a89e5720920d218293)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.207531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.269586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:28.285829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca8e18e118641a89e5720920d218293', 'request_id': 'req-438606be-623e-48d5-8614-7b5f91c445cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:28.296073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca8e18e118641a89e5720920d218293', 'request_id': 'req-438606be-623e-48d5-8614-7b5f91c445cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:28.308826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca8e18e118641a89e5720920d218293', 'request_id': 'req-438606be-623e-48d5-8614-7b5f91c445cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:28.329190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] The token cache is being invalidate because project 0ca8e18e118641a89e5720920d218293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:28.329673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:28.330534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-438606be-623e-48d5-8614-7b5f91c445cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:28.332474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 970/1927] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:28 2022] DELETE /identity/v3/projects/0ca8e18e118641a89e5720920d218293 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:28.339747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.340209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.340533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.369596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:28.370848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:28.446018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 958/1928] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:28.455492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.455790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.456050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.477157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:28.478460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:28.490771 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 971/1929] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:28.496501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.516050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634aafc-977d-40e7-926f-e625151d0c83 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.521511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.522210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.522496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.522780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.523530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:28.523842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.525509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.543507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 959/1930] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:27:28.548224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.558424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d56e8d-d360-4b1c-9ff8-9c56c5f8b95f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.564191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.565176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.565484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.565782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.566442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.568392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.602074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 972/1931] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:28 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:28.612301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.633989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b5f63-b447-4740-9171-7cd51cf16fd1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.704363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.705598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.706117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.706448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.707736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.711434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.740709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 960/1932] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:28 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:28.745692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.757834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea878563-682e-4c87-8e6c-65317dc2e7c6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.763034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.763818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.764110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.764430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.765177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.766875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.778674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 973/1933] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:28.782980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.795736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d26db6-5604-4b71-afff-74f8205ebd2b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.801242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.801947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.802231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.802512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.803254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.804929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404cfa-7c3a-4b90-9f16-7e054ad17e34 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.820687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 961/1934] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:28.822611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:28.834472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e7e4-2651-4418-8639-e838594ab8be admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:28.842092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:28.844083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.845578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.846129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] PATH_INFO: `/v3/projects/234fc2ca7ba7409188389d0af024c509/users/ad83ad2ed75e458d83287c9640997b0b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.881855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=234fc2ca7ba7409188389d0af024c509, user_id=ad83ad2ed75e458d83287c9640997b0b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:28.885840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:28.906982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 974/1935] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:28 2022] PUT /identity/v3/projects/234fc2ca7ba7409188389d0af024c509/users/ad83ad2ed75e458d83287c9640997b0b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:28.918754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:28.919316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:28.919986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:28.963645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] MFA Rules not processed for user `ad83ad2ed75e458d83287c9640997b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:28.965113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:28.981770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:29.069932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 962/1936] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:29.081953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:29.083756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:29.166759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17cdc3-a64c-460e-bb73-71083ebc68bd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:29.172350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:29.173198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:29.173517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:29.173971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:29.185615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:29.187125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0b7d9-6e5c-4037-87a5-12eb5b0025ca service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:29.198010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 975/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:36.412075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:36.413655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:36.530914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abe71f-8e6b-4655-afb3-5b5bb61676a7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:36.540704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:36.542436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:36.542818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:36.543384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] PATH_INFO: `/v3/users/d84cf10772f04fab94b613aadfaf05ac` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:36.546410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84cf10772f04fab94b613aadfaf05ac)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:36.550647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:36.648295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84cf10772f04fab94b613aadfaf05ac'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:36.664384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84cf10772f04fab94b613aadfaf05ac'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:36.687334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84cf10772f04fab94b613aadfaf05ac'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:36.688367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84cf10772f04fab94b613aadfaf05ac'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:36.693806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84cf10772f04fab94b613aadfaf05ac'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:36.713543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 963/1938] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:36 2022] DELETE /identity/v3/users/d84cf10772f04fab94b613aadfaf05ac => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:36.718975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002cd0c-0661-4323-afad-088987e93462 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:36.719556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002cd0c-0661-4323-afad-088987e93462 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:36.719748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002cd0c-0661-4323-afad-088987e93462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:36.746459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1002cd0c-0661-4323-afad-088987e93462 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:36.748243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1002cd0c-0661-4323-afad-088987e93462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:36.900183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 976/1939] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:36.921793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:36.924590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:36.996198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:37.033666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2afb28-b93a-48f3-9d8c-acab28b2a9d5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:37.038243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:37.039121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:37.039332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:37.039587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:37.052064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:37.053900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:37.089587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 964/1940] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:37.470980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002cd0c-0661-4323-afad-088987e93462 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:37.473905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1002cd0c-0661-4323-afad-088987e93462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:37.485325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:37.492821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002cd0c-0661-4323-afad-088987e93462 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:37.501175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:37.504522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:37.505240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:37.506135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] PATH_INFO: `/v3/projects/0a6558beb7204c349dffc9bc625b34f1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:37.509650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6558beb7204c349dffc9bc625b34f1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:37.512956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:37.571469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6558beb7204c349dffc9bc625b34f1', 'request_id': 'req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.579122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6558beb7204c349dffc9bc625b34f1', 'request_id': 'req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.586996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6558beb7204c349dffc9bc625b34f1', 'request_id': 'req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.604132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] The token cache is being invalidate because project 0a6558beb7204c349dffc9bc625b34f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:37.605060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.606085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.611432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 977/1941] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:37 2022] DELETE /identity/v3/projects/0a6558beb7204c349dffc9bc625b34f1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:37.613393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:37.629483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7cd39-2a6f-4054-b891-6eef533aee22 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:37.697102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:37.698020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:37.698456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:37.698724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] PATH_INFO: `/v3/users/1e68cb49f59a46ec98d6d2f5e940c3a9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:37.699752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e68cb49f59a46ec98d6d2f5e940c3a9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:37.701933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:37.761403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e68cb49f59a46ec98d6d2f5e940c3a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.772216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e68cb49f59a46ec98d6d2f5e940c3a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.785640 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e68cb49f59a46ec98d6d2f5e940c3a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.786472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e68cb49f59a46ec98d6d2f5e940c3a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.787173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586de1b3-8f43-49ed-8cb4-4e3338f51a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e68cb49f59a46ec98d6d2f5e940c3a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:37.803745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 965/1942] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:37 2022] DELETE /identity/v3/users/1e68cb49f59a46ec98d6d2f5e940c3a9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:38.117005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.118885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:38.134888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5311ae-e11c-4254-8fb5-834bbb2ea6e6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.218984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.219859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.220133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.220450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] PATH_INFO: `/v3/projects/fa063bb6be7345a2ac6be3c212f6b705` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.221911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa063bb6be7345a2ac6be3c212f6b705)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.223726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.298050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa063bb6be7345a2ac6be3c212f6b705', 'request_id': 'req-d9d1c059-8f70-4423-8d0b-2fc87866c965'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:38.308431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa063bb6be7345a2ac6be3c212f6b705', 'request_id': 'req-d9d1c059-8f70-4423-8d0b-2fc87866c965'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:38.321148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa063bb6be7345a2ac6be3c212f6b705', 'request_id': 'req-d9d1c059-8f70-4423-8d0b-2fc87866c965'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:38.337982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] The token cache is being invalidate because project fa063bb6be7345a2ac6be3c212f6b705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:38.338277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:38.338909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d9d1c059-8f70-4423-8d0b-2fc87866c965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:38.340558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 978/1943] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:38 2022] DELETE /identity/v3/projects/fa063bb6be7345a2ac6be3c212f6b705 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:38.346790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.347171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.347448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.366650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:38.368152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:38.436210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:38.452678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 966/1944] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:38.455380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.457196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.457196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.476287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:38.477537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:38.485948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 979/1945] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:38.490732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.503685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607aeca-dc0b-4a0e-91f0-7f5e1ba39de5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.509209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.509893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.510177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.510447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.511187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:38.511490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.513193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.527056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 967/1946] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:38.532185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.545850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f08f5-854e-46a7-a791-0dcedf962570 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.551953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.552680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.552960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.553234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.553919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.555853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.565423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:38.587815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 980/1947] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:38 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:38.594084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.608628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d08bda-b853-46a1-af85-5501c01a5551 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.695995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.697124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.697124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.697404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.697970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.700198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.727202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 968/1948] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:38 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:38.732176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.743611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb458-13e1-4eda-b030-e1ad13aa8037 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.748835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.749528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.749814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.750093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.750836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.752733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.764382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 981/1949] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:38.768845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.785959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8fada-4dfa-495d-9cdb-b3dad8c7910e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.791425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.792219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.792612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.792983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.793832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.795589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa025d-521b-461c-b8f8-c6694112d196 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.807687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 969/1950] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:38.812477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:38.823635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629296cf-a7cf-42fa-ab8d-d3ff4579c447 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:38.828946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:38.829854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.830164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.830585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] PATH_INFO: `/v3/projects/849e09e8b3884d7482fc9d33287ceb79/users/5ebbb338a8ea426995c61388e98ba3b6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.848084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=849e09e8b3884d7482fc9d33287ceb79, user_id=5ebbb338a8ea426995c61388e98ba3b6, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:38.850841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:38.870485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 982/1951] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:38 2022] PUT /identity/v3/projects/849e09e8b3884d7482fc9d33287ceb79/users/5ebbb338a8ea426995c61388e98ba3b6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:38.885900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:38.886405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:38.886836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:38.927158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] MFA Rules not processed for user `5ebbb338a8ea426995c61388e98ba3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:38.928987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:39.008575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 970/1952] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:39.020446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:39.022389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:39.095878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4325db-6706-4721-adb7-32fcc19192e5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:39.100301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:39.101116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:39.101508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:39.101853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:39.112529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:39.114181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba3326-4af2-4568-a475-ad15cbbcbb68 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:39.124762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 983/1953] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:41.904457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:41.906520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:41.996889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:42.016908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354317f-fbec-4990-b8c1-517a40e6fad4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:42.022932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:42.023857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:42.024143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:42.024471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] PATH_INFO: `/v3/users/f870bc0a641942c6be2917f8f83ba1fd` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:42.025850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f870bc0a641942c6be2917f8f83ba1fd)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:42.027869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:42.094385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f870bc0a641942c6be2917f8f83ba1fd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:42.104580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f870bc0a641942c6be2917f8f83ba1fd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:42.124031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f870bc0a641942c6be2917f8f83ba1fd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:42.124949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f870bc0a641942c6be2917f8f83ba1fd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:42.125582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f870bc0a641942c6be2917f8f83ba1fd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:42.148467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 971/1954] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:41 2022] DELETE /identity/v3/users/f870bc0a641942c6be2917f8f83ba1fd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:42.154921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:42.155830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:42.155830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:42.183033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:42.185064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:42.289456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 984/1955] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:42.304483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:42.306006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:42.401811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374cd8-6e20-4118-9c1c-0cf915ab8218 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:42.409159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:42.409854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:42.410211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:42.410450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:42.423635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:42.426186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:42.442912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 972/1956] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:43.464601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:43.466547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:43.496884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39653bde-5ebe-48af-a713-4813ba963414 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:43.505088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:43.507304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:43.507901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:43.508595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] PATH_INFO: `/v3/projects/eb060f4f653f4c3194430b876587d6a4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:43.510951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb060f4f653f4c3194430b876587d6a4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:43.514839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:43.597801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb060f4f653f4c3194430b876587d6a4', 'request_id': 'req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.600989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:43.607605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb060f4f653f4c3194430b876587d6a4', 'request_id': 'req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.616615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb060f4f653f4c3194430b876587d6a4', 'request_id': 'req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.650744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] The token cache is being invalidate because project eb060f4f653f4c3194430b876587d6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:43.651287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.652728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.659121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 985/1957] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:43 2022] DELETE /identity/v3/projects/eb060f4f653f4c3194430b876587d6a4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:43.660167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:43.671722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e4e2-2c24-4ba3-adec-28544e58e880 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:43.752785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:43.753728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:43.754010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:43.754304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] PATH_INFO: `/v3/users/fc726d5ca3804482873d6b261b919a1b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:43.755668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc726d5ca3804482873d6b261b919a1b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:43.757692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:43.786201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:43.837443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc726d5ca3804482873d6b261b919a1b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.855199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc726d5ca3804482873d6b261b919a1b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.875561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc726d5ca3804482873d6b261b919a1b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.876476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc726d5ca3804482873d6b261b919a1b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.877390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf6bf076-f3ae-41d8-a185-e048c9a1fcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc726d5ca3804482873d6b261b919a1b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:43.894674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 973/1958] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:43 2022] DELETE /identity/v3/users/fc726d5ca3804482873d6b261b919a1b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:44.320134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.322023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:44.347192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fd856-00fd-4e9d-8d24-f1cbc6498ecb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:44.443170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:44.444404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.444765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.445257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] PATH_INFO: `/v3/projects/3efd4a142f15430dad4acdd4c48e7d28` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.446822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efd4a142f15430dad4acdd4c48e7d28)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:44.449355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:44.511872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efd4a142f15430dad4acdd4c48e7d28', 'request_id': 'req-40f8d97e-bdad-407d-adf8-81cfec158a8c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:44.518887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efd4a142f15430dad4acdd4c48e7d28', 'request_id': 'req-40f8d97e-bdad-407d-adf8-81cfec158a8c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:44.526959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efd4a142f15430dad4acdd4c48e7d28', 'request_id': 'req-40f8d97e-bdad-407d-adf8-81cfec158a8c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:44.542786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] The token cache is being invalidate because project 3efd4a142f15430dad4acdd4c48e7d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:44.543084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:44.543799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40f8d97e-bdad-407d-adf8-81cfec158a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:44.545514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 986/1959] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:44 2022] DELETE /identity/v3/projects/3efd4a142f15430dad4acdd4c48e7d28 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:44.552523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.553269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.553832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.576669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:44.578027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:44.651401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 974/1960] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:44.660234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.660846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.661197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.685532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:44.686894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:44.697787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 987/1961] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:44.702339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.717671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecf122-3aeb-42b3-8694-953cee0b492e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:44.726951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:44.728280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.728807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.729331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.730672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:44.731191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:44.733835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:44.748462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 975/1962] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:44.752890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.762521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142ee18-674a-4818-b5a8-6a2048f80741 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:44.767558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:44.768276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.768582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.768866 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.769576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:44.771526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:44.812085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 988/1963] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:44 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:44.816956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.827670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3257a8-7ab9-4d5a-9e92-784392cf54ba admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:44.906937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:44.907643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.907944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.908352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.909150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:44.911326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:44.936366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 976/1964] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:44 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:44.941152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.957072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023de1d-5701-4bbb-bc9f-e5f5733483ba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:44.963994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:44.964740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:44.965019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:44.965289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:44.966037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:44.967905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:44.985016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 989/1965] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:44.986318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:44.998917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904923-6c13-453a-8231-b304ad014911 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.005438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.006260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.006622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.006963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.007791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.010692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3ed36-2d16-44bb-80e6-88e3b44bbe60 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.019783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:45.032082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.032904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 977/1966] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:45.050797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577809a-f0e2-4a2a-8a87-7c653ea8b9ca admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.056958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.058050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.058412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.058768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] PATH_INFO: `/v3/projects/e0f05a3c7435459484df541e9e14e89c/users/aeec074381be4ae48a7cd088802fb762/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.081635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f05a3c7435459484df541e9e14e89c, user_id=aeec074381be4ae48a7cd088802fb762, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.084450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.104459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 990/1967] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:45 2022] PUT /identity/v3/projects/e0f05a3c7435459484df541e9e14e89c/users/aeec074381be4ae48a7cd088802fb762/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:45.114385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.114873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.115363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.164565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] MFA Rules not processed for user `aeec074381be4ae48a7cd088802fb762`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:45.167749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:45.295747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 978/1968] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:45.301631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.313923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:45.438273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518477c4-cd39-4dca-b43e-5233b6b71ca2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.446078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.446768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.447048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.447323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.448191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.450396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.521652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 991/1969] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:45 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:45.522447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.539747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a8fff-66e6-47d4-bbb1-f94f2eb62a8e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.621884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.623049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.623527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.623886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.624941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.627132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.655185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 979/1970] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:45 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:45.656550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.670578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6d058-d4b3-4e4d-9b88-d36943275f71 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.676903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.677609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.677900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.678678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.678913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.680719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.700561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 992/1971] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:45.704979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.721490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56050c1-d7e1-40ef-9054-d6e9a37f2e92 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:45.750969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:45.752712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:45.753114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:45.753511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] PATH_INFO: `/v3/projects/f6fdecbd24284243a5c9afb4bab90626/users/8d95212d5ffb4a0b8f3de20bbb435657/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:45.793344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fdecbd24284243a5c9afb4bab90626, user_id=8d95212d5ffb4a0b8f3de20bbb435657, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:45.797169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:45.826152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 980/1972] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:45 2022] PUT /identity/v3/projects/f6fdecbd24284243a5c9afb4bab90626/users/8d95212d5ffb4a0b8f3de20bbb435657/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:45.830815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:45.846839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2ac9c-ffeb-4107-b614-630e7fa4552d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:46.011741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:46.012671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:46.013019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:46.013575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:46.014978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:46.019373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487a534-1151-4fda-9e7d-c520e3700468 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:46.035559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 993/1973] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:46.040585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:46.054874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209296f-5ad2-41b9-a6e9-b8f7ec73910d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:46.061677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:46.062634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:46.062912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:46.063201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] PATH_INFO: `/v3/projects/f6fdecbd24284243a5c9afb4bab90626/users/8d95212d5ffb4a0b8f3de20bbb435657/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:46.065599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fdecbd24284243a5c9afb4bab90626, user_id=8d95212d5ffb4a0b8f3de20bbb435657, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:46.068613 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:46.080855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 981/1974] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:46 2022] PUT /identity/v3/projects/f6fdecbd24284243a5c9afb4bab90626/users/8d95212d5ffb4a0b8f3de20bbb435657/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:46.091919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:46.092213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:46.092601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:46.125047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] MFA Rules not processed for user `8d95212d5ffb4a0b8f3de20bbb435657`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:46.126908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:46.254788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 994/1975] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:46.272159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:46.275266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:46.338530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:46.367675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2028c-a186-476d-a880-a00339d4af97 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:46.375156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:46.376377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:46.377092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:46.377683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:46.393051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:46.394913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fb766-2a26-44d1-bf04-4228a82d4335 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:46.414400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 982/1976] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:46.931821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:46.939535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:47.017911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c78fa-a240-4efd-8742-cca88a927efa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:47.022653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:47.023549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:47.023827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:47.024094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] PATH_INFO: `/v3/users/aeec074381be4ae48a7cd088802fb762` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:47.025338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeec074381be4ae48a7cd088802fb762)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:47.027278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:47.087185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeec074381be4ae48a7cd088802fb762'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.087892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeec074381be4ae48a7cd088802fb762'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.088530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeec074381be4ae48a7cd088802fb762'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.096531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeec074381be4ae48a7cd088802fb762'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.106338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeec074381be4ae48a7cd088802fb762'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.138081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 995/1977] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:46 2022] DELETE /identity/v3/users/aeec074381be4ae48a7cd088802fb762 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:47.144107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:47.144767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:47.145113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:47.166266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:47.167535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:47.237965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 983/1978] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:47.248144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:47.249864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:47.314284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558b09f-79d8-47a3-964f-f630e5571fea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:47.318147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:47.318819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:47.319099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:47.319374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:47.329985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:47.331494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:47.342619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 996/1979] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:47.651852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:47.653879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:47.667927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a963013-7d7a-4909-8f2d-1240986bfc10 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:47.675464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:47.676011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:47.676011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:47.676587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] PATH_INFO: `/v3/projects/e0f05a3c7435459484df541e9e14e89c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:47.678175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f05a3c7435459484df541e9e14e89c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:47.680265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:47.739206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f05a3c7435459484df541e9e14e89c', 'request_id': 'req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.746914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f05a3c7435459484df541e9e14e89c', 'request_id': 'req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.757498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f05a3c7435459484df541e9e14e89c', 'request_id': 'req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.771731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] The token cache is being invalidate because project e0f05a3c7435459484df541e9e14e89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:47.772711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.773480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.775149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 984/1980] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:47 2022] DELETE /identity/v3/projects/e0f05a3c7435459484df541e9e14e89c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:47.779565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:47.792804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9d98-7fea-45db-ad72-8ac0416dc426 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:47.821134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:47.890191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:47.891545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:47.891838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:47.892117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] PATH_INFO: `/v3/users/8d95212d5ffb4a0b8f3de20bbb435657` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:47.893953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d95212d5ffb4a0b8f3de20bbb435657)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:47.896395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:47.961941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d95212d5ffb4a0b8f3de20bbb435657'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.962637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d95212d5ffb4a0b8f3de20bbb435657'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.963269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d95212d5ffb4a0b8f3de20bbb435657'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.973046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d95212d5ffb4a0b8f3de20bbb435657'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:47.981658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1556e9b0-8459-4fda-9abd-23655ada1437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d95212d5ffb4a0b8f3de20bbb435657'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.001671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 997/1981] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:47 2022] DELETE /identity/v3/users/8d95212d5ffb4a0b8f3de20bbb435657 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:48.363155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:48.365364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:48.381563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f36e96-9c87-4dd9-a6aa-6b59a32917ec admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:48.453949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:48.455988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.456540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.457050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] PATH_INFO: `/v3/projects/f6fdecbd24284243a5c9afb4bab90626` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.459340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fdecbd24284243a5c9afb4bab90626)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:48.463271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:48.484248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:48.538060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fdecbd24284243a5c9afb4bab90626', 'request_id': 'req-fddfbd48-b9a1-403b-af45-b8b957f743b3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.545157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fdecbd24284243a5c9afb4bab90626', 'request_id': 'req-fddfbd48-b9a1-403b-af45-b8b957f743b3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.556768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fdecbd24284243a5c9afb4bab90626', 'request_id': 'req-fddfbd48-b9a1-403b-af45-b8b957f743b3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.574817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] The token cache is being invalidate because project f6fdecbd24284243a5c9afb4bab90626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:48.575271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.576033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fddfbd48-b9a1-403b-af45-b8b957f743b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:48.577914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 985/1982] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:48 2022] DELETE /identity/v3/projects/f6fdecbd24284243a5c9afb4bab90626 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:48.584254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.584574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.584872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.607033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:48.608392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:48.680215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 998/1983] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:48.690554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.690863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.691142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.725713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:48.727000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:48.735427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 986/1984] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:48.740799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:48.758512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fefbd-a805-4d03-9536-9bbc37c40210 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:48.763780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:48.764517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.764795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.765062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.765800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:48.766102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:48.767890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:48.781294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 999/1985] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:27:48.786824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:48.800019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d8a65-54c4-470d-a38a-87fb9daddb99 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:48.807756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:48.808903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.809760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.810250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.811553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:48.815372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:48.854804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 987/1986] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:48.861259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:48.875469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a79ef-667c-4c71-a9aa-d721126630a0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:48.913576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:48.965943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:48.967026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:48.967026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:48.967307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:48.968022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:48.970453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.026812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1000/1987] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:48 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:49.034076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.057682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b1525-b743-41c8-864d-0da514e4130c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.063637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.064858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.064858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.065183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.065972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.069668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.081235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 988/1988] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:49.085096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.099347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17890-106b-4b6f-b1e8-ed934dfdccc3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.106392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.107224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.107655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.107992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.108829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.110505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432d801-f5bf-4fc2-9f9f-52de5bca90bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.123904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1001/1989] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:49.128748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.154590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd1fa5-ef10-49f9-b5cf-150f41c35767 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.164077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.165232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.165599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.165938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] PATH_INFO: `/v3/projects/9fa28152e8934511bd5111b8929cadaf/users/068e0e0e5337433db2a4a8f403229736/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.190019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa28152e8934511bd5111b8929cadaf, user_id=068e0e0e5337433db2a4a8f403229736, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.194311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.216903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 989/1990] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:49 2022] PUT /identity/v3/projects/9fa28152e8934511bd5111b8929cadaf/users/068e0e0e5337433db2a4a8f403229736/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:49.229972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.230270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.230566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.275214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] MFA Rules not processed for user `068e0e0e5337433db2a4a8f403229736`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:49.276984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:49.392810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1002/1991] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:49.398176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.493078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949dbe-bacb-4c56-8232-9f1bd005055f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.509438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.510638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.511081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.511377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.512394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.517283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.564950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 990/1992] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:49 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:49.571661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.591197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdf92c-f2a3-4420-ae28-72f7bc0fd8d4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.723267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.724175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.726175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.726748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.727584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.733767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.795998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1003/1993] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:49 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:49.801238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.831965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95a697-16e8-475f-b1a6-851fad1da7b5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.849331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.850082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.850376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.850655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.851413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.862808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.876376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 991/1994] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:49.879709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.891927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbe89c-1c42-45a4-aef8-e0d407492491 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.897899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:49.898828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:49.899107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:49.899382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] PATH_INFO: `/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33/users/556cd34225b4445fa1ae504f84e4d977/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:49.925747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3238cffb8f34efb8a3dce94b87e9a33, user_id=556cd34225b4445fa1ae504f84e4d977, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:49.929090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:49.951389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1004/1995] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:49 2022] PUT /identity/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33/users/556cd34225b4445fa1ae504f84e4d977/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:49.957699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:49.980597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6decd-1131-40dd-94d8-d2ca09ba7c7c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:49.992679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:50.093418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:50.096710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:50.098886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:50.099495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:50.102951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:50.106200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba25855-6ecf-4b21-896f-d4b5d48aeb63 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:50.132673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 992/1996] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:50.139514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:50.150058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:50.155916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271badf-e67d-46cd-bc31-59c6b90db9af admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:50.163518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:50.164648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:50.165292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:50.165684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] PATH_INFO: `/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33/users/556cd34225b4445fa1ae504f84e4d977/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:50.168466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3238cffb8f34efb8a3dce94b87e9a33, user_id=556cd34225b4445fa1ae504f84e4d977, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:50.171720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:50.214242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1005/1997] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:50 2022] PUT /identity/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33/users/556cd34225b4445fa1ae504f84e4d977/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:50.218423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:50.218868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:50.219258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:50.269034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] MFA Rules not processed for user `556cd34225b4445fa1ae504f84e4d977`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:50.270761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:50.401920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 993/1998] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:50.460592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:50.463712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:50.605983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0534d-70c7-47ba-88eb-b82491664614 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:50.614914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:50.617385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:50.617921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:50.618457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:50.646572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:50.649656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e31ae7-6410-4fce-bee1-ab5459e081d7 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:50.672842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1006/1999] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:50.904708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:50.906406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.009609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a3f1e-252d-4578-a490-b0cb769cd2f3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:51.015474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:51.016863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.017240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.017974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] PATH_INFO: `/v3/users/ff8c3de0c861494393d8517c092062dc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.019955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8c3de0c861494393d8517c092062dc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:51.022475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:51.078822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:51.094897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8c3de0c861494393d8517c092062dc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.104730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8c3de0c861494393d8517c092062dc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.123706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8c3de0c861494393d8517c092062dc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.124727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8c3de0c861494393d8517c092062dc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.126066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8c3de0c861494393d8517c092062dc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.146846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 994/2000] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:50 2022] DELETE /identity/v3/users/ff8c3de0c861494393d8517c092062dc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:51.153807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.154457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.154718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.178469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:51.179899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.260416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1007/2001] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:51.271810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:51.274066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.342285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9f273-456a-44be-87d3-64af7956ae50 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:51.355211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:51.356126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.356647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.357171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.372712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:51.374701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1ae57-7a98-4af3-88e3-6befbd3faf5c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:51.388831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 995/2002] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:51.488990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:51.491910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.511386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce21ddc-6aa3-48b8-8f5a-ba1b92da9841 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:51.519775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:51.521299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.521753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.522225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] PATH_INFO: `/v3/users/068e0e0e5337433db2a4a8f403229736` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.524507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068e0e0e5337433db2a4a8f403229736)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:51.528664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:51.669194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068e0e0e5337433db2a4a8f403229736'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.671610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068e0e0e5337433db2a4a8f403229736'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.675418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068e0e0e5337433db2a4a8f403229736'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.689020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068e0e0e5337433db2a4a8f403229736'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.707244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068e0e0e5337433db2a4a8f403229736'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:51.739783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1008/2003] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:51 2022] DELETE /identity/v3/users/068e0e0e5337433db2a4a8f403229736 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:51.746039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3080c-3891-4100-94ad-771690adef3b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.747244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3080c-3891-4100-94ad-771690adef3b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.747697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3080c-3891-4100-94ad-771690adef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.771822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-38c3080c-3891-4100-94ad-771690adef3b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:51.773492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-38c3080c-3891-4100-94ad-771690adef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.789116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:51.791579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.820936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:51.862522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 996/2004] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:51.875591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3080c-3891-4100-94ad-771690adef3b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:51.878557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-38c3080c-3891-4100-94ad-771690adef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:51.943494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bac973-a072-4aad-b1e2-69720d417d03 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:51.949837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:51.950828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:51.951100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:51.951529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] PATH_INFO: `/v3/projects/c1bff836d1da41c9b5bb85edd595cd4a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:51.953121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1bff836d1da41c9b5bb85edd595cd4a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:51.956063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.008967 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3080c-3891-4100-94ad-771690adef3b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.020273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:52.032853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1bff836d1da41c9b5bb85edd595cd4a', 'request_id': 'req-eaff4232-cb5d-473b-b777-27460a7de6af'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.044891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1bff836d1da41c9b5bb85edd595cd4a', 'request_id': 'req-eaff4232-cb5d-473b-b777-27460a7de6af'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.056644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1bff836d1da41c9b5bb85edd595cd4a', 'request_id': 'req-eaff4232-cb5d-473b-b777-27460a7de6af'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.086088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] The token cache is being invalidate because project c1bff836d1da41c9b5bb85edd595cd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:52.086601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.088114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eaff4232-cb5d-473b-b777-27460a7de6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.090726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1009/2005] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:51 2022] DELETE /identity/v3/projects/c1bff836d1da41c9b5bb85edd595cd4a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:52.098713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c4f1-c1bc-4a4d-b97f-38d79a6306e5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.099163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c4f1-c1bc-4a4d-b97f-38d79a6306e5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.099627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c4f1-c1bc-4a4d-b97f-38d79a6306e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.121887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.122941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.123419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.123938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.127024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.138287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cce4c4f1-c1bc-4a4d-b97f-38d79a6306e5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:52.139622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cce4c4f1-c1bc-4a4d-b97f-38d79a6306e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.209176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1010/2006] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:52.210187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.213111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.218376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.218697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.219039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.233092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 997/2007] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:52.243310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:52.245470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.263928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1011/2008] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:27:52.265269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.275534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f71a23-d702-435f-b600-e31b876f1d3c service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.281015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.281513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.281795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.282068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.282811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:52.283310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.285360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.299816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 998/2009] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:52.304592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.315985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222f919-db1f-4e0b-92da-9f8499fd8ceb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.322938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.323625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.324000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.324308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.324933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.327176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.360365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1012/2010] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:52 2022] POST /identity/v3/projects => generated 359 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:52.365400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.382079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8d9a-e5cf-4f84-99f4-002c280539a7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.462302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.463186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.463660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.464109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.465721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.468350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.499200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 999/2011] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:52 2022] POST /identity/v3/users => generated 280 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:52.504272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.515956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97116a95-9384-46bf-b02a-920644c81d5a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.525437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.526150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.526440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.526726 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.527524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.529608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.532433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:52.546127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1013/2012] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:27:52.554809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.563419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.567686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.579446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e5174-5a51-4a94-be03-69c646805f85 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.591996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.593404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.593987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.594541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.596860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.600730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.605404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4144-83b8-420a-966e-96f10a46ecca admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.611602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.612602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.612966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.613313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] PATH_INFO: `/v3/projects/9fa28152e8934511bd5111b8929cadaf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.614753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa28152e8934511bd5111b8929cadaf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.616846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.620192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1000/2013] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:52.625523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.638087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d87d5-c291-48e2-88e4-c9b28397dd87 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.644235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.645184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.645468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.645741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] PATH_INFO: `/v3/projects/99700fcae6f94192bf65a070fb4b6e60/users/78ca8d4956934f7a860ea68e3747d95b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.669194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99700fcae6f94192bf65a070fb4b6e60, user_id=78ca8d4956934f7a860ea68e3747d95b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.671964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.685215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:52.695985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa28152e8934511bd5111b8929cadaf', 'request_id': 'req-cfeeb90a-6ed9-4e9e-a360-1401320895d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.696816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1001/2014] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:52 2022] PUT /identity/v3/projects/99700fcae6f94192bf65a070fb4b6e60/users/78ca8d4956934f7a860ea68e3747d95b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:52.698073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.700773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.711627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa28152e8934511bd5111b8929cadaf', 'request_id': 'req-cfeeb90a-6ed9-4e9e-a360-1401320895d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.718288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa28152e8934511bd5111b8929cadaf', 'request_id': 'req-cfeeb90a-6ed9-4e9e-a360-1401320895d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.731521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] The token cache is being invalidate because project 9fa28152e8934511bd5111b8929cadaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:52.731808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.732490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cfeeb90a-6ed9-4e9e-a360-1401320895d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.734106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1014/2015] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:52 2022] DELETE /identity/v3/projects/9fa28152e8934511bd5111b8929cadaf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:27:52.738863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.739148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.739424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.776206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] MFA Rules not processed for user `78ca8d4956934f7a860ea68e3747d95b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:52.777475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.799041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7991b2-5cd1-4f44-a1de-acdc6b23e919 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.804627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.805540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.805812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.806090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] PATH_INFO: `/v3/users/5ebbb338a8ea426995c61388e98ba3b6` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.807804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ebbb338a8ea426995c61388e98ba3b6)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.809759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.854896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1015/2016] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:52.856288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.857708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.866744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ebbb338a8ea426995c61388e98ba3b6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.875493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ebbb338a8ea426995c61388e98ba3b6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.879995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb081-3caf-4e56-a343-c3ad61128703 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.885883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:52.886535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:52.886814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:52.887091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] PATH_INFO: `/v3/users/556cd34225b4445fa1ae504f84e4d977` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:52.888348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=556cd34225b4445fa1ae504f84e4d977)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:52.890322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:52.892550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ebbb338a8ea426995c61388e98ba3b6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.893490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ebbb338a8ea426995c61388e98ba3b6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.894280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ebbb338a8ea426995c61388e98ba3b6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.908023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1002/2017] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:52 2022] DELETE /identity/v3/users/5ebbb338a8ea426995c61388e98ba3b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:52.909036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:52.910615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:52.948231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556cd34225b4445fa1ae504f84e4d977'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.949085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556cd34225b4445fa1ae504f84e4d977'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.949843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556cd34225b4445fa1ae504f84e4d977'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.959125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556cd34225b4445fa1ae504f84e4d977'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.971112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556cd34225b4445fa1ae504f84e4d977'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:52.975212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:52.997010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d994f-82c6-4815-8bb9-6f7552b31814 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:52.999776 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1016/2018] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:52 2022] DELETE /identity/v3/users/556cd34225b4445fa1ae504f84e4d977 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:53.000796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.002327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.007177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.008836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.009364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.009869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] PATH_INFO: `/v3/users/ad83ad2ed75e458d83287c9640997b0b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.012370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad83ad2ed75e458d83287c9640997b0b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.014448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.041808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:53.083128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad83ad2ed75e458d83287c9640997b0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.087664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28820c-9de3-4367-a62a-f4f6f29590ce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.092886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad83ad2ed75e458d83287c9640997b0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.094727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.096034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.096767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.097279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.100609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.113532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad83ad2ed75e458d83287c9640997b0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.115015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad83ad2ed75e458d83287c9640997b0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.115849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e4490604-5dc0-4e47-87cf-df3bd86790de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad83ad2ed75e458d83287c9640997b0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.136235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1003/2019] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:52 2022] DELETE /identity/v3/users/ad83ad2ed75e458d83287c9640997b0b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:53.137884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.138313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.138718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.176441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:53.179288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.219799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.222606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5f85-db50-4a3f-8d1f-43dcaa02b728 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.238203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1017/2020] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:52 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:53.242625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.242938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.243213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.278740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:53.280197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.315800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1004/2021] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:53.317060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.318621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.343088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad491204-cc1c-469c-850f-e2cb3cb785ad None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.350263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.351621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.351956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.352341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] PATH_INFO: `/v3/users/78ca8d4956934f7a860ea68e3747d95b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.354361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ca8d4956934f7a860ea68e3747d95b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.356839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.394519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1018/2022] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:53.398745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.400519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.433341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ca8d4956934f7a860ea68e3747d95b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.443261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ca8d4956934f7a860ea68e3747d95b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.459056 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:53.465330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ca8d4956934f7a860ea68e3747d95b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.466427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ca8d4956934f7a860ea68e3747d95b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.466892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ca8d4956934f7a860ea68e3747d95b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.485465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1005/2023] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:27:53 2022] DELETE /identity/v3/users/78ca8d4956934f7a860ea68e3747d95b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:53.485948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.488648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.495650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801b702-8df0-46b7-b680-223687f86c2d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.513170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:53.609541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.610664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.611283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.611810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.615308 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.620198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7aee0-6a78-49e9-a96c-d7a04dfde188 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.624885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.626045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.626344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.626879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.629828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.727977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.731466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63642ba7-f6ec-4dc9-88b7-a38951f7be06 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.751581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1006/2024] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:53.754861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.755376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.755868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.760716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.763733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.776623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1019/2025] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:53.777747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.780538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.789604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:53.791597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:53.796779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25503d49-9c53-44f8-a513-d33bda902a39 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.804872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1007/2026] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:53.806111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.807094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.807311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.807586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] PATH_INFO: `/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.808876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3238cffb8f34efb8a3dce94b87e9a33)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.811415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.842403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:53.862532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29a6ee-2de0-4bba-8d9b-f6f99f64f4f7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:53.867576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:53.868544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.869201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.869414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.886864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:53.888959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b14a3f-b0fd-4355-bd6f-7a7a379b9afe service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:53.904215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1008/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:53.905281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3238cffb8f34efb8a3dce94b87e9a33', 'request_id': 'req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.919507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3238cffb8f34efb8a3dce94b87e9a33', 'request_id': 'req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.930131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3238cffb8f34efb8a3dce94b87e9a33', 'request_id': 'req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.952902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] The token cache is being invalidate because project b3238cffb8f34efb8a3dce94b87e9a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:53.953536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.954474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fd9f8abc-b81e-48f0-b6e4-cf9d390fd670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:53.956974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1020/2028] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:53 2022] DELETE /identity/v3/projects/b3238cffb8f34efb8a3dce94b87e9a33 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:53.963676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:53.963964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:53.964247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:53.991154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:53.992926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:54.117473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1009/2029] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:54.131292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.131913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.132570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.185928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:54.188210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:54.216618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1021/2030] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:54.218011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.233743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9744f4-cd3a-4c1f-9921-4a16dde0fe56 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.257653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.258474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.258831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.259118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.259895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:54.260210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.261983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.330347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1010/2031] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:54.332118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.352464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d962b-7eee-4106-ad58-3d7a34f66722 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.358258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.358965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.359250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.359544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.360708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.362425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.414008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:54.444387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1022/2032] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:54 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:27:54.450748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.470546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b7b2f-c9d4-4b76-9fb9-ecb5c3002d99 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.525178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:54.595593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.596465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.597671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.598191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.598628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:54.598767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.600024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.603789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.618572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa91be-a6af-4529-b20a-6c647540fd4b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.630636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.631471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.632062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.632715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] PATH_INFO: `/v3/projects/234fc2ca7ba7409188389d0af024c509` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.634877 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234fc2ca7ba7409188389d0af024c509)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.637921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.675748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1011/2033] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:54 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:54.681251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.699203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff8ba3-40d7-4063-a240-9970e203c6be admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.712653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.712653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.712653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.712653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.717071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.717071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.731913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1012/2034] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:54.735373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.740301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:54.778204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a2de2-4db5-4f20-88c6-da40b51d6e6f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.790503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234fc2ca7ba7409188389d0af024c509', 'request_id': 'req-adb84333-4459-4703-8d62-4002e8840b20'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.806585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234fc2ca7ba7409188389d0af024c509', 'request_id': 'req-adb84333-4459-4703-8d62-4002e8840b20'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.824913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234fc2ca7ba7409188389d0af024c509', 'request_id': 'req-adb84333-4459-4703-8d62-4002e8840b20'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.851766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] The token cache is being invalidate because project 234fc2ca7ba7409188389d0af024c509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:54.852404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.852991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:54.853899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.854043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:54.854441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:54.854702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] PATH_INFO: `/v3/projects/849e09e8b3884d7482fc9d33287ceb79` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:54.856054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849e09e8b3884d7482fc9d33287ceb79)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:54.857448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1023/2035] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:54 2022] DELETE /identity/v3/projects/234fc2ca7ba7409188389d0af024c509 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:54.859111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:54.860527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:54.864217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:54.922995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84333-4459-4703-8d62-4002e8840b20 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:54.943225 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849e09e8b3884d7482fc9d33287ceb79', 'request_id': 'req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.957326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849e09e8b3884d7482fc9d33287ceb79', 'request_id': 'req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:54.986637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849e09e8b3884d7482fc9d33287ceb79', 'request_id': 'req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.014098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] The token cache is being invalidate because project 849e09e8b3884d7482fc9d33287ceb79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:55.014797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.015756 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cb2ffe33-414f-47a5-b21a-dcedbfa7b8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.017693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1013/2036] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:54 2022] DELETE /identity/v3/projects/849e09e8b3884d7482fc9d33287ceb79 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:55.020019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f715-511b-4a9b-a6c2-77dbc7bcabe1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.020572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f715-511b-4a9b-a6c2-77dbc7bcabe1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.021040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78f715-511b-4a9b-a6c2-77dbc7bcabe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.027199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.027914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.028198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.028596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.029281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.030989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.046401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1024/2037] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:55.046896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2f78f715-511b-4a9b-a6c2-77dbc7bcabe1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.047628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.049606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2f78f715-511b-4a9b-a6c2-77dbc7bcabe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.052381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.062858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1014/2038] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.064358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.064828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.065163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.073060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872aaaa-1bbe-4b3a-92eb-00225704bea1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.078795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.079696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.080196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.080481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] PATH_INFO: `/v3/projects/99700fcae6f94192bf65a070fb4b6e60` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.082936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99700fcae6f94192bf65a070fb4b6e60)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.087029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.098677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.100779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.115400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1015/2039] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.117470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.119900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.143229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8598fd-3191-4f4f-a246-573d6f18a6da None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.162705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99700fcae6f94192bf65a070fb4b6e60', 'request_id': 'req-13faf85f-eee3-447d-b6f0-e8117b8c2039'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.169709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99700fcae6f94192bf65a070fb4b6e60', 'request_id': 'req-13faf85f-eee3-447d-b6f0-e8117b8c2039'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.178212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99700fcae6f94192bf65a070fb4b6e60', 'request_id': 'req-13faf85f-eee3-447d-b6f0-e8117b8c2039'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.182307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:55.190360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:55.196872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] The token cache is being invalidate because project 99700fcae6f94192bf65a070fb4b6e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:27:55.197552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.199166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-13faf85f-eee3-447d-b6f0-e8117b8c2039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:27:55.202398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1025/2040] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:27:55 2022] DELETE /identity/v3/projects/99700fcae6f94192bf65a070fb4b6e60 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:55.208824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdadde0-1c3f-4ac9-92e3-0f3cee098047 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.209827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdadde0-1c3f-4ac9-92e3-0f3cee098047 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.210476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdadde0-1c3f-4ac9-92e3-0f3cee098047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.235627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0cdadde0-1c3f-4ac9-92e3-0f3cee098047 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.237583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0cdadde0-1c3f-4ac9-92e3-0f3cee098047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.246327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.247255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.247698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.247972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] PATH_INFO: `/v3/projects/12d99dadc4d34ff2aff5da28e6cd4c51/users/7c59e1317b914d618a487b069525a16d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.266430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d99dadc4d34ff2aff5da28e6cd4c51, user_id=7c59e1317b914d618a487b069525a16d, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.269319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c64bc-02dc-434c-b906-c6264a335bd1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.286891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1016/2041] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:55 2022] PUT /identity/v3/projects/12d99dadc4d34ff2aff5da28e6cd4c51/users/7c59e1317b914d618a487b069525a16d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:55.292131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671826d-592b-4a1b-b4b9-7ee50aab7421 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.292364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671826d-592b-4a1b-b4b9-7ee50aab7421 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.292787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671826d-592b-4a1b-b4b9-7ee50aab7421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.313794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8671826d-592b-4a1b-b4b9-7ee50aab7421 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.316846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8671826d-592b-4a1b-b4b9-7ee50aab7421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.331368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1026/2042] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.336489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.337430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.337855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.358273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.359546 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.400985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1017/2043] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.403423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.403944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.404479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.441062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1027/2044] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.441372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.443670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.445858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.453718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6882654-babe-4c7a-9359-2f78922b3c64 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.459390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.460079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.460398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.460523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1018/2045] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.461032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.461917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:55.462273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.464074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148dca0-a10e-410f-a4e4-623147452a47 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.466958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.485947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaee6d-a7b1-4402-b680-a42d117814cd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.495200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.497194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.497588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.498029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.498638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1028/2046] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:55.499821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:55.499821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.501881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.502080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.502695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.503133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.526298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1019/2047] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:55.526298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.526298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.539004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:55.541139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:55.560220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1029/2048] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:55.560220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.572765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f374c-cec9-4c7d-8410-635775c43a4b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.579442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.581446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.581446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.581446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.582479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.584641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.605142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4798-4ee3-4a13-942c-492cfcc56e42 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.608835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.609494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.609776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.610048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.617272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1030/2049] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/projects => generated 395 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:55.617272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.620007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.621698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.632965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1020/2050] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:55.633772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.650081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60964-19dc-42e0-ba58-c1474042fa6a service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.687867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:55.690347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f8119-8040-48fc-ab19-cfb3ce105d1d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.696507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.697293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.697614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.698326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.699202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.701206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.705982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:55.737378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.737583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1031/2051] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:55.737668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.737757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.737831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.737831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.737831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:27:55.737831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.737831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.753147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1021/2052] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:27:55.753812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.766697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff90bc-5e25-49da-aabb-f616aa8b0f4f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.832884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5106-8eaf-47ac-89e0-1d1af1c68754 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.833307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.834096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.834365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.834690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.835400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.837379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.845790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.848307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.848624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.848955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.849948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.855236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.892123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1022/2053] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:55.893147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.904027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1032/2054] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:55.904027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:55.917345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278bb90-0eca-406e-a8bb-765b4c1733fc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.923436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.924414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.924874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.925296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.926157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.928859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.938284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ab829-e874-446a-9d71-519359acbe55 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:55.955193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:55.956796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:55.958283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:55.958545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:55.963636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:55.966686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:55.993556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1023/2055] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:55 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:55.994626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1033/2056] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:55.995584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.004549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.008859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472bf0e-a8d5-4135-b6f5-1b7906cf3b91 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.195575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.196682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.197078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.197461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.198438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.200959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.213017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5278fa-44b2-43f8-842b-7cd22fbb288f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.216563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1034/2057] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:56.217703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.234668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aebf8b-8c13-48c4-835b-cb7ea4878dc9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.239365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.241733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.242694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.243158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.243534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.244655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.245422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.245981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.246629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.248645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.249336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.254007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.279598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1035/2058] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:56 2022] POST /identity/v3/users => generated 286 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:56.281437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.285846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1024/2059] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:56.286789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.296508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890688a-849e-4bcd-95cc-d5612ad75100 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.302471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.303294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.303700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.304058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.304865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.308484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812ddae-05be-48da-9c9c-52cc7399e879 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.312407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.315239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.315584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.315680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.316015 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.316911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.318718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.334512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1036/2060] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:56.334512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.334919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1025/2061] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:56.336524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.348392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85f1f9-bc3d-44d4-9e9a-d3517fe16a90 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.350537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4bbec-5761-4f30-8b2a-bdc5db67c67a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.354182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.355206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.355390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.355753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.356647 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.358980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.360126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.362644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.362980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.363416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] PATH_INFO: `/v3/projects/2fbb3724d74d4ff3b428f89f29abe3a5/users/207c5a7cae0147e09c47ac4b24d2cc87/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.374301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1026/2062] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:56.378281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.391553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21181e3-f83c-443f-bf21-ce016e5e7223 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.393208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbb3724d74d4ff3b428f89f29abe3a5, user_id=207c5a7cae0147e09c47ac4b24d2cc87, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.398299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.399002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.399238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.399501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.399777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] PATH_INFO: `/v3/projects/0a2740ace1dc4b1b99e77fa28d72c78e/users/ec70c25644d541c1915d76f20d908638/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.403931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:56.423805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2740ace1dc4b1b99e77fa28d72c78e, user_id=ec70c25644d541c1915d76f20d908638, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.427210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1037/2063] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:56 2022] PUT /identity/v3/projects/2fbb3724d74d4ff3b428f89f29abe3a5/users/207c5a7cae0147e09c47ac4b24d2cc87/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:56.427434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e9a2e-d1e9-40e0-85d6-038fc6612048 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.427878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.442594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:56.451601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1027/2064] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:56 2022] PUT /identity/v3/projects/0a2740ace1dc4b1b99e77fa28d72c78e/users/ec70c25644d541c1915d76f20d908638/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:56.453217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42b63-eb7e-4f8e-ad2c-5e75902939bf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.453581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42b63-eb7e-4f8e-ad2c-5e75902939bf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.453958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a42b63-eb7e-4f8e-ad2c-5e75902939bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.506174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-62a42b63-eb7e-4f8e-ad2c-5e75902939bf None None] MFA Rules not processed for user `207c5a7cae0147e09c47ac4b24d2cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:56.507327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-62a42b63-eb7e-4f8e-ad2c-5e75902939bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.564247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7285dc-2081-4c7f-9a37-576f7c4f0f88 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.640911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1028/2065] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:56.642020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.642340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.643556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.655141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.655720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.656077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.656437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] PATH_INFO: `/v3/projects/b99a389f8cd34552ae1b3c30803669ae/users/b9d9cb0073b5452698e549792adf6733/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.677451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99a389f8cd34552ae1b3c30803669ae, user_id=b9d9cb0073b5452698e549792adf6733, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:56.682429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedd344-80da-40ac-bc15-bcb2f36b8138 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:56.682690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] MFA Rules not processed for user `ec70c25644d541c1915d76f20d908638`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:56.683698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.699112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1038/2066] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:56 2022] PUT /identity/v3/projects/b99a389f8cd34552ae1b3c30803669ae/users/b9d9cb0073b5452698e549792adf6733/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:56.701883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e281-c9d9-40ac-a070-6349a19d0672 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.702178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e281-c9d9-40ac-a070-6349a19d0672 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.702442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e281-c9d9-40ac-a070-6349a19d0672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.740598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1dd0e281-c9d9-40ac-a070-6349a19d0672 None None] MFA Rules not processed for user `7c59e1317b914d618a487b069525a16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:56.742555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1dd0e281-c9d9-40ac-a070-6349a19d0672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.780535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1029/2067] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:56.781242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.782705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.834700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1039/2068] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:56.836624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.837186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.837685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.858284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf063f3-8efd-443e-b373-61628e663701 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.864795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:56.864795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.882104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:56.893023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:56.952251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.952975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.953296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.953541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:56.962462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1040/2069] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:56.964062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:56.968505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:56.985250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ff885-9d7a-4850-a65e-9aefcc164be0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:56.990157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:56.990839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:56.991122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:56.991412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.035731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.037392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.047953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1030/2070] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:57.049086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.051052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:57.070179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270c720-5c21-4c78-8650-924ccec3276c service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.070416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.072447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.076307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.077034 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.077585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.077864 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.078576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.082459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.085420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1041/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:57.086906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.123752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1031/2072] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:27:57 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:57.135477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.227558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d95c-67a4-48d6-8638-200c2dd99be4 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.302175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae1f4b-90a8-47e4-afc9-fcb19e7d9d6a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.323836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.325259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.325683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.326064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.327048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.331932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.379243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1032/2073] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:27:57 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:27:57.380920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.386784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.388027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.389624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.390188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.402982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c99fb9-4aa7-4b58-9f34-437fa41f3e61 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.420933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.425735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.458991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.461367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.462246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.462246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.463263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1042/2074] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:27:57.466450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.474284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.520047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1033/2075] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:57.537903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.552989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298308a-44b9-4537-a156-5cc332d884d8 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.564094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.565291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.565577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.565851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] PATH_INFO: `/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f/users/55d13f9190124270a69860e77ad50247/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.600064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e098d1fcb0b647cdb0d9bd21cd01b27f, user_id=55d13f9190124270a69860e77ad50247, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.607463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.632312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:57.645853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1043/2076] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:57 2022] PUT /identity/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f/users/55d13f9190124270a69860e77ad50247/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:57.651359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.667302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c0d4-09b2-470d-a215-3ece114d237f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.761178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:27:57.846439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.847127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.847416 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.847701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.848524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.850243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ee0d-544c-4ad4-a207-e9341c15672d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:57.890840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1034/2077] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:27:57.896608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:57.945380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e03b14-1f60-4c28-9c06-432cd12d05b8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:57.957515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:57.959136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:57.959628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:57.960162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] PATH_INFO: `/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f/users/55d13f9190124270a69860e77ad50247/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:57.967202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e098d1fcb0b647cdb0d9bd21cd01b27f, user_id=55d13f9190124270a69860e77ad50247, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:57.975206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:58.005770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1044/2078] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:27:57 2022] PUT /identity/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f/users/55d13f9190124270a69860e77ad50247/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:27:59.078390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225685b-cb79-4c0d-8db9-54ed68c1accc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:59.079560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225685b-cb79-4c0d-8db9-54ed68c1accc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:59.080411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225685b-cb79-4c0d-8db9-54ed68c1accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:59.171550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a225685b-cb79-4c0d-8db9-54ed68c1accc None None] MFA Rules not processed for user `b9d9cb0073b5452698e549792adf6733`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:59.173331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a225685b-cb79-4c0d-8db9-54ed68c1accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:59.330138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1035/2079] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:59.343835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:59.369136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:59.369990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:59.370970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:59.457675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] MFA Rules not processed for user `55d13f9190124270a69860e77ad50247`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:27:59.459974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:27:59.474058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a3bfa-9fac-46db-9382-52cee3d869b8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:59.484003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:59.484755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:59.485046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:59.485320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:59.505641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:59.507131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:59.533669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1045/2080] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:27:59.664950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1036/2081] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:27:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:27:59.720186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:27:59.740242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e04ec-3a57-4d53-a691-e1fb57028642 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:27:59.754513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:27:59.756052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:27:59.756861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:27:59.757571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:27:59.903710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:27:59.907476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ebe3ab-1265-451b-9e98-d335c00a299c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:27:59.927450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1046/2082] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:27:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:00.125738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:00.193180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:00.222578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809fec-49ed-45c6-a8e2-5344c50b3330 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:00.228995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:00.230091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:00.231178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:00.231178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] PATH_INFO: `/v3/users/207c5a7cae0147e09c47ac4b24d2cc87` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:00.232470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=207c5a7cae0147e09c47ac4b24d2cc87)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:00.234886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:00.316787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207c5a7cae0147e09c47ac4b24d2cc87'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:00.332948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207c5a7cae0147e09c47ac4b24d2cc87'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:00.353626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207c5a7cae0147e09c47ac4b24d2cc87'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:00.354595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207c5a7cae0147e09c47ac4b24d2cc87'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:00.355328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207c5a7cae0147e09c47ac4b24d2cc87'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:00.379054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1037/2083] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:00 2022] DELETE /identity/v3/users/207c5a7cae0147e09c47ac4b24d2cc87 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:00.383895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:00.384810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:00.384810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:00.408538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:00.410631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:00.584333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1047/2084] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:00.645044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:00.646624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:00.815410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb961d9-94cf-4c2d-ba7d-c89df611a4e9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:00.821961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:00.822868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:00.823265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:00.823664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:00.841982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:00.849202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b1aa6-c27c-4816-9ff2-0d6325491832 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:00.857126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1038/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:28:01.214610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:01.217425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:01.250353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b1683-2ef9-4bb1-966b-814d96415513 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:01.259333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:01.260417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:01.260690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:01.260931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] PATH_INFO: `/v3/projects/2fbb3724d74d4ff3b428f89f29abe3a5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:01.262655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbb3724d74d4ff3b428f89f29abe3a5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:01.265163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:01.310948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:01.339583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbb3724d74d4ff3b428f89f29abe3a5', 'request_id': 'req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:01.346992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbb3724d74d4ff3b428f89f29abe3a5', 'request_id': 'req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:01.354875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbb3724d74d4ff3b428f89f29abe3a5', 'request_id': 'req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:01.376361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] The token cache is being invalidate because project 2fbb3724d74d4ff3b428f89f29abe3a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:01.377091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:01.378212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9e06a8e4-8846-42e2-ac9d-276c08b6f63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:01.381112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1048/2086] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:01 2022] DELETE /identity/v3/projects/2fbb3724d74d4ff3b428f89f29abe3a5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:02.868487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:02.869054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:02.869054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:02.872840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1039/2087] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:28:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:28:02.884885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:02.885749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:02.886188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:02.930100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] MFA Rules not processed for user `26b59442ce5949a59dfe58d1b4b98e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:02.931334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:02.957508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:02.962134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.050768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1049/2088] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:03.067012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:03.069231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.084978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e6da-ec0f-4720-b271-575b9fd4c716 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:03.095084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:03.096823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:03.096823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:03.097089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:03.109484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:03.111731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804021ba-fdf4-4824-b328-63d4d76bcf96 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:03.143145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1050/2089] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:03.172903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea604012-6e87-4d21-81cf-87d862bd38f0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:03.181782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:03.182995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:03.183307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:03.183735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] PATH_INFO: `/v3/users/5e2597ff867544f9953c519bf29a9bfa` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:03.186086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2597ff867544f9953c519bf29a9bfa)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:03.188219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:03.193216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:03.299720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2597ff867544f9953c519bf29a9bfa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.316651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2597ff867544f9953c519bf29a9bfa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.324849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2597ff867544f9953c519bf29a9bfa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.325726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2597ff867544f9953c519bf29a9bfa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.326419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2597ff867544f9953c519bf29a9bfa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.328933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:03.329327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:03.329766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:03.344980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1040/2090] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:02 2022] DELETE /identity/v3/users/5e2597ff867544f9953c519bf29a9bfa => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:03.350227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] MFA Rules not processed for user `b9d9cb0073b5452698e549792adf6733`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:03.351455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.425256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1051/2091] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:03.624160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:03.626223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.730529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba20e7-bf35-44ef-996c-1cb6dca31cfe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:03.739995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:03.741662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:03.742163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:03.742644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] PATH_INFO: `/v3/projects/05e46da777a54cbe92cbfb00c37d8bb5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:03.745108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e46da777a54cbe92cbfb00c37d8bb5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:03.748888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:03.835616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e46da777a54cbe92cbfb00c37d8bb5', 'request_id': 'req-53084b4c-cdde-4504-b099-29f8d3c72248'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.843674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e46da777a54cbe92cbfb00c37d8bb5', 'request_id': 'req-53084b4c-cdde-4504-b099-29f8d3c72248'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.857782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e46da777a54cbe92cbfb00c37d8bb5', 'request_id': 'req-53084b4c-cdde-4504-b099-29f8d3c72248'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.872068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:03.874632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.874838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] The token cache is being invalidate because project 05e46da777a54cbe92cbfb00c37d8bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:03.875340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.876204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-53084b4c-cdde-4504-b099-29f8d3c72248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:03.879043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1041/2092] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:03 2022] DELETE /identity/v3/projects/05e46da777a54cbe92cbfb00c37d8bb5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:03.886030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda317-43fe-4702-8324-41af1c853f02 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:03.886315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda317-43fe-4702-8324-41af1c853f02 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:03.886747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda317-43fe-4702-8324-41af1c853f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:03.907399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-57eda317-43fe-4702-8324-41af1c853f02 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:03.909266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-57eda317-43fe-4702-8324-41af1c853f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:03.917383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:03.968080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:03.995163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2548c-581a-4656-98e0-cb1e97451fb5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.024254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1042/2093] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:28:04.034118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.034808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.035586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.075490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.076404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.076794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.077174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.079328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:04.080481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:04.084609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:04.104481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1043/2094] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:04.110788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.131493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1815-77b5-4da3-b864-685e80720db1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.139552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.140835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.141520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.142040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.143983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:28:04.144661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.148229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.167972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.169556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.182061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1052/2095] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:04.191051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1044/2096] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:28:04.196158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.216913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bd1f2-5fc0-4d56-8cc5-39cbbf1ccb92 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.223141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.223983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.224347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.224631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.225429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.228245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.272695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1053/2097] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:04 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:04.277720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.306074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67a230-5673-4064-8979-53f04dc545bd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.445482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.447471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.448133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.448746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.449933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.453484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.496156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1045/2098] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:04 2022] POST /identity/v3/users => generated 291 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:04.501160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.517551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857d21e-354f-4ebb-bf48-a51fca19f4a9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.532532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.533364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.533704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.534043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.534854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.547178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.566529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1054/2099] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:04.572515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.590141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589277f-f067-45f7-bcb7-a0f3d0ccf622 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.596617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.597648 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.598152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.598606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.599750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.601814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d61521-20c3-4a47-9533-575bb8f9a943 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.617095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1046/2100] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:04.621416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.635685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190c086-f824-4b30-80cb-c7dd5be55afa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.645516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.647469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.648226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.648942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] PATH_INFO: `/v3/projects/dff1983b1b3047378b88d87d54aa709d/users/23abc809ab644c1da0270a35ac8bab21/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.686673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff1983b1b3047378b88d87d54aa709d, user_id=23abc809ab644c1da0270a35ac8bab21, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.689983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.708115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1055/2101] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:04 2022] PUT /identity/v3/projects/dff1983b1b3047378b88d87d54aa709d/users/23abc809ab644c1da0270a35ac8bab21/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:04.719468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.719941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.720691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.750719 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:04.752698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:04.830518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1047/2102] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:04.844762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:04.848551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:04.929199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:04.934645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7075e1-10a2-4090-9f73-5ad155d46e33 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:04.941123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:04.941835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:04.942149 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:04.942530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:04.953571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:04.955511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2075f5-fa91-499b-8db2-f69c329d343a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:04.966836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1056/2103] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:06.598936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:06.600638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:06.635768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26b218-600d-4754-bcea-58b5dd46f0a7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:06.643612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:06.645828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:06.645828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:06.645828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] PATH_INFO: `/v3/users/ec70c25644d541c1915d76f20d908638` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:06.646666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec70c25644d541c1915d76f20d908638)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:06.649533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:06.655944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:06.720007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec70c25644d541c1915d76f20d908638'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:06.734293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec70c25644d541c1915d76f20d908638'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:06.774075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec70c25644d541c1915d76f20d908638'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:06.775399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec70c25644d541c1915d76f20d908638'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:06.776288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec70c25644d541c1915d76f20d908638'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:06.797187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1048/2104] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:06 2022] DELETE /identity/v3/users/ec70c25644d541c1915d76f20d908638 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:06.803113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:06.805119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:06.805652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:06.844779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:06.846921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:06.997577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1057/2105] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:07.039638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:07.041600 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:07.115098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf947cd-5ac9-49fb-a505-375a43b7b3ac admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:07.119953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:07.120995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:07.121583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:07.122078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:07.141162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:07.145040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a76333-f8fd-4f6f-9004-0311008fd8a2 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:07.164571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1049/2106] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:28:08.035091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.043022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:08.072693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a202fa8-4f04-4065-87c8-5fb533a59455 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.078510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.079421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.079850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.080298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] PATH_INFO: `/v3/projects/0a2740ace1dc4b1b99e77fa28d72c78e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.081796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2740ace1dc4b1b99e77fa28d72c78e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.083791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.162495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2740ace1dc4b1b99e77fa28d72c78e', 'request_id': 'req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:08.172568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2740ace1dc4b1b99e77fa28d72c78e', 'request_id': 'req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:08.182672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2740ace1dc4b1b99e77fa28d72c78e', 'request_id': 'req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:08.200374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] The token cache is being invalidate because project 0a2740ace1dc4b1b99e77fa28d72c78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:08.200670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:08.201366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-497d590b-3ed5-4ea1-9a4c-dea18a1dc8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:08.202984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1058/2107] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:08 2022] DELETE /identity/v3/projects/0a2740ace1dc4b1b99e77fa28d72c78e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:08.209539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.211173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.211173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.235192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:08.237092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:08.324941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1050/2108] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:08.337083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.337384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.337680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.370728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:08.373627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:08.391097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1059/2109] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:08.398744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.417835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7635a-0c14-42f4-82fa-ea2f96fb12d1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.432942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.434350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.434736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.435121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.436019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:28:08.436469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.438587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.459302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1051/2110] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:28:08.466051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.487016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a955b10-70af-44a6-b627-e319d81693eb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.498900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.501463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.502080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.502843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.504201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.515151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.574243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1060/2111] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:08 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:28:08.576968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.608972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa4ebc-2f64-425e-86b1-2df8bfbc5b55 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.645656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:08.745644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.746348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.746627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.746899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.747825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.749898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.782548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1052/2112] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:08 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:28:08.785522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.817199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10c21a-a485-4fb2-b835-18de94f86b2f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.829102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.832963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.832963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.833585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.835593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.840609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.880147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.881401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1061/2113] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:08.898689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1d404-1c5a-4dcf-ad79-09304997a45b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.908504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.910308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.911195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.911876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:08.915618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:08.917295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cef9f-d809-4d2a-aaf9-30b2d5520968 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:08.952878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:08.964105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1053/2114] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:08.974335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4087cb9-eb58-4af7-8103-dafef948e673 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:08.983423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:08.985282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:08.985883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:08.986907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] PATH_INFO: `/v3/projects/52c96385e67941648eee6bf532b3f141/users/d6682b6bd10a4609b68db0bd02f193b6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:09.048094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c96385e67941648eee6bf532b3f141, user_id=d6682b6bd10a4609b68db0bd02f193b6, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:09.060415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:09.098543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1062/2115] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:08 2022] PUT /identity/v3/projects/52c96385e67941648eee6bf532b3f141/users/d6682b6bd10a4609b68db0bd02f193b6/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:09.108706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43183-dc5b-478e-ad89-27d40330be79 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:09.109320 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43183-dc5b-478e-ad89-27d40330be79 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:09.109826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d43183-dc5b-478e-ad89-27d40330be79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:09.150464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-85d43183-dc5b-478e-ad89-27d40330be79 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:09.152670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-85d43183-dc5b-478e-ad89-27d40330be79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:09.423165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1054/2116] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:09.439369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:09.442554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:09.514574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:09.671547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926c93-540e-41e8-aa81-6486ef0fbc08 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:09.681810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:09.683139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:09.683634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:09.684520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:09.718163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:09.726288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:09.761975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1063/2117] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:15.037032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a70c8-b809-4fc9-896d-dc93d7fe7de4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:15.037485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a70c8-b809-4fc9-896d-dc93d7fe7de4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:15.037703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a70c8-b809-4fc9-896d-dc93d7fe7de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:15.078435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-522a70c8-b809-4fc9-896d-dc93d7fe7de4 None None] MFA Rules not processed for user `23abc809ab644c1da0270a35ac8bab21`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:15.080393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-522a70c8-b809-4fc9-896d-dc93d7fe7de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:15.187395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1055/2118] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:15.203882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:15.205839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:15.340372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77855b1-dab3-4cfe-9752-4d16c79af7c3 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:15.348040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:15.349423 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:15.349796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:15.350084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:15.365888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:15.370573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c39538a-7209-4a10-8386-f0b69808eab5 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:15.390828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1064/2119] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:20.099493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:20.099989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:20.100627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:20.145841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] MFA Rules not processed for user `23abc809ab644c1da0270a35ac8bab21`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:20.147592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:20.164455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1056/2120] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:21.757804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:21.759177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:21.759734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:21.835517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] MFA Rules not processed for user `d6682b6bd10a4609b68db0bd02f193b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:21.838698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:22.078977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1065/2121] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:22.080184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.137073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5635479-7264-4bf7-835c-3b555a696444 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.146553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.148470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.149179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.149800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:22.152180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:22.156156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:22.217590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:22.246800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1057/2122] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:22 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:22.256346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.319378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb992-ad53-45fa-b2ee-76643c601059 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.372251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:22.443883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.444671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.444961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.445234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:22.445963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:22.448912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:22.489072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1066/2123] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:22 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:22.493899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.520204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8901d-5036-4968-9346-378a4f614676 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.537386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.538535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.541985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.542269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:22.545732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:22.550913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:22.571778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1058/2124] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:22.578573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.600537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7217abb-9f53-4252-b276-54b960ec7710 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.635243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.638458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.640087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.640431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] PATH_INFO: `/v3/projects/adf8b075ad134044b7c840b7c53e41e6/users/c8a163cbe29d49aeb63b4cfa8186fe8e/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:22.686964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf8b075ad134044b7c840b7c53e41e6, user_id=c8a163cbe29d49aeb63b4cfa8186fe8e, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:22.694037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:22.731183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1067/2125] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:22 2022] PUT /identity/v3/projects/adf8b075ad134044b7c840b7c53e41e6/users/c8a163cbe29d49aeb63b4cfa8186fe8e/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:22.733349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.761936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8847d-c1b2-4fcf-8c68-22f7bf9e2b5b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.937052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.937966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.938263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.938759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:22.942030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:22.946080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:22.973445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1059/2126] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:22.973855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:22.995677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684bef1-4a6c-43ac-97ba-22341a18fabd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:22.998159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:22.999117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:22.999555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:22.999844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] PATH_INFO: `/v3/projects/adf8b075ad134044b7c840b7c53e41e6/users/c8a163cbe29d49aeb63b4cfa8186fe8e/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:23.002597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf8b075ad134044b7c840b7c53e41e6, user_id=c8a163cbe29d49aeb63b4cfa8186fe8e, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:23.005529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:23.035659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1068/2127] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:22 2022] PUT /identity/v3/projects/adf8b075ad134044b7c840b7c53e41e6/users/c8a163cbe29d49aeb63b4cfa8186fe8e/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:23.414295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:23.418046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:23.463306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a89d7-2f46-4f39-ada8-9e5a3242d27d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:23.621367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:23.622458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:23.622814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:23.623152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] PATH_INFO: `/v3/users/7c59e1317b914d618a487b069525a16d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:23.624736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c59e1317b914d618a487b069525a16d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:23.627219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:23.716889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c59e1317b914d618a487b069525a16d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:23.728426 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c59e1317b914d618a487b069525a16d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:23.770798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c59e1317b914d618a487b069525a16d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:23.776256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c59e1317b914d618a487b069525a16d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:23.780907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c59e1317b914d618a487b069525a16d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:23.810880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:23.829177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1060/2128] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:23 2022] DELETE /identity/v3/users/7c59e1317b914d618a487b069525a16d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:24.695357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:24.700249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:24.747596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd71231-8d86-46b7-afe2-9c2e079a65bd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:24.989207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:24.990134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:24.990420 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:24.990700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] PATH_INFO: `/v3/projects/12d99dadc4d34ff2aff5da28e6cd4c51` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:24.992583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d99dadc4d34ff2aff5da28e6cd4c51)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:25.002803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:25.159432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d99dadc4d34ff2aff5da28e6cd4c51', 'request_id': 'req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.175190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d99dadc4d34ff2aff5da28e6cd4c51', 'request_id': 'req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.185399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d99dadc4d34ff2aff5da28e6cd4c51', 'request_id': 'req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.212270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] The token cache is being invalidate because project 12d99dadc4d34ff2aff5da28e6cd4c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:25.213000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.214262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.217681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1069/2129] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:24 2022] DELETE /identity/v3/projects/12d99dadc4d34ff2aff5da28e6cd4c51 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:25.221964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:25.235990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0b3a0-8082-451b-8611-224f6f723fd1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:25.357966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:25.362344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:25.363993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:25.364453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] PATH_INFO: `/v3/users/55d13f9190124270a69860e77ad50247` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:25.366671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d13f9190124270a69860e77ad50247)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:25.385504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:25.475862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d13f9190124270a69860e77ad50247'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.490735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d13f9190124270a69860e77ad50247'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.530425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d13f9190124270a69860e77ad50247'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.531821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d13f9190124270a69860e77ad50247'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.532731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-376f00c7-2dcc-4115-868a-728960d15524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d13f9190124270a69860e77ad50247'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:25.568695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1061/2130] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:25 2022] DELETE /identity/v3/users/55d13f9190124270a69860e77ad50247 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:26.011337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:26.013681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:26.039699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8ff03-e7ab-4209-b37c-5e1357a89eba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:26.057307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:26.277189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:26.278416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.279990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.280519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] PATH_INFO: `/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.288730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e098d1fcb0b647cdb0d9bd21cd01b27f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:26.291111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:26.426954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e098d1fcb0b647cdb0d9bd21cd01b27f', 'request_id': 'req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:26.442253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e098d1fcb0b647cdb0d9bd21cd01b27f', 'request_id': 'req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:26.456738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e098d1fcb0b647cdb0d9bd21cd01b27f', 'request_id': 'req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:26.526162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] The token cache is being invalidate because project e098d1fcb0b647cdb0d9bd21cd01b27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:26.526481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:26.527438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37fe7009-ff9f-4dcb-b69a-8f9b87dea618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:26.529727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1070/2131] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:26 2022] DELETE /identity/v3/projects/e098d1fcb0b647cdb0d9bd21cd01b27f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:26.537948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.538585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.538881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.577504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:26.578809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:26.717802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1062/2132] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:26.732461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.733262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.733504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.765264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:26.766884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:26.777555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1071/2133] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:26.782007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:26.796215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cc14f-ab7f-499c-acf8-39f432d0bb7d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:26.802623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:26.803377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.804371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.804371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.804832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:28:26.805508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:26.807018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1ee12-dd01-4d3a-b200-b8125621dcd9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:26.829134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1063/2134] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:28:26.833808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:26.838055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.838385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.838771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.845180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0461a94-ea47-4d1a-877f-c3c5925612ee None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:26.853003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:26.856395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:26.857299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:26.857584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:26.858394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:26.868604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:26.871879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:26.872590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] MFA Rules not processed for user `b9d9cb0073b5452698e549792adf6733`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:26.874872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:26.941275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1072/2135] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:26 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:26.943074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:27.044570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1064/2136] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:27.174444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb2cef-4094-4f1c-aee8-e7071c7c2654 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:27.239224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:27.240769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:27.247653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:27.251085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:27.255677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:27.260019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:27.328833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1073/2137] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:26 2022] POST /identity/v3/users => generated 287 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:27.333670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:27.354291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3edfb-5e81-47a7-8fde-e9b4a8064ba7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:27.375637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:27.376989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:27.377321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:27.377736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:27.378727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:27.380743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:27.415891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1065/2138] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:27.417744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:27.436968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:27.460174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea5f6f-d59b-43a0-9b65-061778c73814 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:27.484079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:27.487943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:27.488978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:27.489573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:27.494936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:27.509504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ea974-8e94-4bf4-a531-7657645c7475 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:27.562069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1074/2139] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:27.567071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:27.587129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688afdf5-1825-42e4-843c-a5ae0fb959bc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:27.596685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:27.597637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:27.597925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:27.598201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] PATH_INFO: `/v3/projects/011515b93c744af9b0c60b59b2b2dc81/users/0dfccbc1b8cf42808bd04af306f717d1/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:27.637324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011515b93c744af9b0c60b59b2b2dc81, user_id=0dfccbc1b8cf42808bd04af306f717d1, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:27.640151 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:27.692499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:27.692797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:27.693077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:27.694909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1066/2140] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:27 2022] PUT /identity/v3/projects/011515b93c744af9b0c60b59b2b2dc81/users/0dfccbc1b8cf42808bd04af306f717d1/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:27.754005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] MFA Rules not processed for user `0dfccbc1b8cf42808bd04af306f717d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:27.756475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:28.020284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1075/2141] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:28.024824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.181739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bd93d-3133-4aed-a356-0af8d7156715 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.190844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.192770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.193424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.193708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.196112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.201178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.249302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1067/2142] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:28 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:28.250096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.269442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24554b-f8e5-4690-a9af-92df20c45986 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.373373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.375370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.375370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.375370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.375370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.378551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.404792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1076/2143] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:28 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:28.408478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.424575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3ea76-13bc-471b-98cf-7380a0d4c794 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.430021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.430890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.431293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.431674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.432585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.434771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.452352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1068/2144] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:28.453643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.466998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d764-9c45-4cce-bda2-59f9135bde80 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.477408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.479172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.479708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.480193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] PATH_INFO: `/v3/projects/065ebd719975437dbd1611033fe8e016/users/894fb8c2de9443629d965c0a6fc0cf43/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.505592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=065ebd719975437dbd1611033fe8e016, user_id=894fb8c2de9443629d965c0a6fc0cf43, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.508644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.525106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1077/2145] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:28 2022] PUT /identity/v3/projects/065ebd719975437dbd1611033fe8e016/users/894fb8c2de9443629d965c0a6fc0cf43/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:28.528979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.540541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d831ac-13ea-496f-afbe-510edef38b47 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.603298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:28.618552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.619234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.619518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.619794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.620568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.622282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d2ae7-646f-4cea-87b5-d8eb1270ab12 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.636375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1069/2146] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:28.638947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.652157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87b911-2bd3-4f8b-9717-9477ac8256aa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:28.658249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:28.659584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.660079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.660637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] PATH_INFO: `/v3/projects/065ebd719975437dbd1611033fe8e016/users/894fb8c2de9443629d965c0a6fc0cf43/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.663777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065ebd719975437dbd1611033fe8e016, user_id=894fb8c2de9443629d965c0a6fc0cf43, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:28.667228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:28.694025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1078/2147] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:28 2022] PUT /identity/v3/projects/065ebd719975437dbd1611033fe8e016/users/894fb8c2de9443629d965c0a6fc0cf43/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:28.694752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:28.695096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:28.695498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:28.771543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] MFA Rules not processed for user `894fb8c2de9443629d965c0a6fc0cf43`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:28.774458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:28.921160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1070/2148] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:28.933398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:28.935599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:29.004396 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:29.086496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d19c28-da2a-4b6b-8303-d48f36a71fb8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:29.100905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:29.103506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:29.104005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:29.105106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:29.326332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:29.329105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603aaa4-b253-47e3-8209-ec84cd319043 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:29.354049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1079/2149] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:32.365863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:32.489818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909b249-40bf-4571-b8e5-4409f71b472d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:32.497555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:32.499494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:32.501817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:32.503136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] PATH_INFO: `/v3/users/0dfccbc1b8cf42808bd04af306f717d1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:32.505966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfccbc1b8cf42808bd04af306f717d1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:32.509836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:32.614992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfccbc1b8cf42808bd04af306f717d1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:32.634680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfccbc1b8cf42808bd04af306f717d1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:32.666221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:32.676946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfccbc1b8cf42808bd04af306f717d1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:32.687637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfccbc1b8cf42808bd04af306f717d1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:32.687637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfccbc1b8cf42808bd04af306f717d1'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:32.721086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1071/2150] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:32 2022] DELETE /identity/v3/users/0dfccbc1b8cf42808bd04af306f717d1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:32.722372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:32.723032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:32.723032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:32.752397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:32.753701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:32.884769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1080/2151] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:32.897098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:32.899930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:33.030721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52647e60-2a1f-49d3-98e2-7614ca6e42f9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:33.065309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:33.066025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:33.066305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:33.066580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:33.099462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:33.101345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:33.117128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1072/2152] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:34.480270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:34.483696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:34.530208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d43a-a3f8-4f45-a290-a0c01bbaf774 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:34.542072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:34.544406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:34.545037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:34.563577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] PATH_INFO: `/v3/projects/011515b93c744af9b0c60b59b2b2dc81` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:34.569526 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011515b93c744af9b0c60b59b2b2dc81)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:34.571591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:34.701870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011515b93c744af9b0c60b59b2b2dc81', 'request_id': 'req-6650b33b-d731-4b32-9405-b7fd9b3f76a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:34.724042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011515b93c744af9b0c60b59b2b2dc81', 'request_id': 'req-6650b33b-d731-4b32-9405-b7fd9b3f76a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:34.747534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011515b93c744af9b0c60b59b2b2dc81', 'request_id': 'req-6650b33b-d731-4b32-9405-b7fd9b3f76a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:34.775514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] The token cache is being invalidate because project 011515b93c744af9b0c60b59b2b2dc81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:34.775934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:34.777450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6650b33b-d731-4b32-9405-b7fd9b3f76a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:34.779584 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1081/2153] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:34 2022] DELETE /identity/v3/projects/011515b93c744af9b0c60b59b2b2dc81 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:34.787799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:34.818610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8251-56cc-4459-aa7a-91918f230822 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:35.025548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:35.027085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:35.027654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:35.028235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] PATH_INFO: `/v3/users/894fb8c2de9443629d965c0a6fc0cf43` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:35.032571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=894fb8c2de9443629d965c0a6fc0cf43)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:35.038199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:35.139540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894fb8c2de9443629d965c0a6fc0cf43'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:35.161717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894fb8c2de9443629d965c0a6fc0cf43'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:35.193263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894fb8c2de9443629d965c0a6fc0cf43'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:35.194361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894fb8c2de9443629d965c0a6fc0cf43'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:35.195578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894fb8c2de9443629d965c0a6fc0cf43'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:35.237145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1073/2154] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:34 2022] DELETE /identity/v3/users/894fb8c2de9443629d965c0a6fc0cf43 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:35.249946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c921-bb50-496f-a889-78d23ba958cc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:35.250742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c921-bb50-496f-a889-78d23ba958cc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:35.251596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c921-bb50-496f-a889-78d23ba958cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:35.335166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:35.378661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-acf4c921-bb50-496f-a889-78d23ba958cc None None] MFA Rules not processed for user `c8a163cbe29d49aeb63b4cfa8186fe8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:35.380730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-acf4c921-bb50-496f-a889-78d23ba958cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:35.605069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1082/2155] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:35.620461 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:35.623057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:35.679582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:35.880246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94e298-4c24-4aa5-9a74-0d1b6fd99eeb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:35.890367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:35.891596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:35.893133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:35.893714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:35.897983 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:36.152298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:36.156133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:36.206566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1074/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:28:36.447887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:36.449080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:36.449887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:36.517808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:36.523594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:36.692586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1083/2157] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:28:36.784156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:36.785729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:36.875746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:36.878653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:37.068057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347f9d2-92d7-41bc-9f19-be2b4714e54e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:37.082282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:37.083377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:37.083661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:37.083951 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] PATH_INFO: `/v3/projects/065ebd719975437dbd1611033fe8e016` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:37.090646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=065ebd719975437dbd1611033fe8e016)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:37.093209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:37.108960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcd295-2df6-4202-a187-b3c579b5c0bc service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:37.122518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:37.123202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:37.123491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:37.123769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:37.140834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:37.146408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e25693-2ef2-436d-a2c8-1a870ba6d8d1 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:37.163981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1075/2158] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:28:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:28:37.211086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:37.223744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065ebd719975437dbd1611033fe8e016', 'request_id': 'req-23c10c74-7a94-4875-87a6-401ac0d928ec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:37.246604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065ebd719975437dbd1611033fe8e016', 'request_id': 'req-23c10c74-7a94-4875-87a6-401ac0d928ec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:37.255259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065ebd719975437dbd1611033fe8e016', 'request_id': 'req-23c10c74-7a94-4875-87a6-401ac0d928ec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:37.289144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] The token cache is being invalidate because project 065ebd719975437dbd1611033fe8e016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:37.290522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:37.291749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-23c10c74-7a94-4875-87a6-401ac0d928ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:37.293743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1084/2159] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:36 2022] DELETE /identity/v3/projects/065ebd719975437dbd1611033fe8e016 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:37.314520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:37.317522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:37.318204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:37.356496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:37.359517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:37.491936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:37.586381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1076/2160] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:37.589329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596a25-e409-4504-860d-11137b365a2e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:37.590077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596a25-e409-4504-860d-11137b365a2e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:37.590804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596a25-e409-4504-860d-11137b365a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:37.782384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-20596a25-e409-4504-860d-11137b365a2e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:37.791915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-20596a25-e409-4504-860d-11137b365a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:37.839425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1085/2161] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:37.844759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:37.883090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c730d-674b-4f93-8b66-b37d83b9f893 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:37.910395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:37.910925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:37.911398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:37.911853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:37.913202 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:28:37.913713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:37.931352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:38.008656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1077/2162] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:28:38.010547 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596a25-e409-4504-860d-11137b365a2e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:38.032821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596a25-e409-4504-860d-11137b365a2e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:38.087492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:38.091309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:38.092904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:38.095040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:38.100249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:38.100249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:38.227691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1086/2163] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:38 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:38.232786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:38.245691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0d92d-6792-43b6-ab8a-d3af00ec6e4f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:38.415172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:38.415995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:38.416393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:38.417234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:38.421353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:38.424845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:38.537841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1078/2164] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:38 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:38.539351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:38.563332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da08d83-ca14-41a4-8bdc-e537fee9dd3a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:38.598113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:38.604792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:38.605219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:38.605529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:38.611201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:38.628340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:38.710153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1087/2165] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:38.716259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:38.750596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4e91-3ae1-4922-bb12-043011c68333 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:38.789386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:38.790628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:38.793325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:38.793720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:38.794581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:38.798248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1e51-f440-4338-8a96-9ffbabbcc98d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:38.828278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1079/2166] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:38.833258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:38.870685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5516d-cea7-4e53-8635-e9626d4bd699 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:38.915845 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:38.934297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:38.935025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:38.935473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] PATH_INFO: `/v3/projects/d1134cd659be4d9a9ea186e99239a81b/users/1594c4449e19424e9f0163763bd1a57c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:39.021164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1134cd659be4d9a9ea186e99239a81b, user_id=1594c4449e19424e9f0163763bd1a57c, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:39.024034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:39.088468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1088/2167] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:38 2022] PUT /identity/v3/projects/d1134cd659be4d9a9ea186e99239a81b/users/1594c4449e19424e9f0163763bd1a57c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:39.093620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:39.093940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:39.094197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:39.131818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] MFA Rules not processed for user `1594c4449e19424e9f0163763bd1a57c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:39.133745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:39.247185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:39.255225 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1080/2168] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:39.641227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c49b3-20c0-4001-b895-b8f57735875b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:39.713616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:39.714338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:39.714629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:39.714934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:39.715647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:39.723792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:39.830739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:39.857605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1089/2169] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:39 2022] POST /identity/v3/projects => generated 385 bytes in 612 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:39.907260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f8fa9-7842-417b-a194-a0f7afd2fc86 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:39.995188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:40.009616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:40.010695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:40.011097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:40.011674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:40.012568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:40.014841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:40.079739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1081/2170] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:39 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:40.080551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:40.124494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbf1ce-db39-4b65-8fe1-ac8c64079de8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:40.154385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:40.155543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:40.156119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:40.156581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:40.157919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:40.165159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:40.213794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1090/2171] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:28:40.218386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:40.254308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885233a6-8f3b-4702-b0d2-a24806a87073 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:40.276087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:40.278081 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:40.279105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:40.279543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:40.280562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:40.283487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18e2d5-3a16-4d94-acec-2646a3a1ba2a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:40.301770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1082/2172] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:40.311568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:40.348493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794f53-f647-45e3-b04f-0bda1673bbec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:40.366860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:40.375474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:40.376075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:40.376591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] PATH_INFO: `/v3/projects/003fa09ccf014b76ad53b88fa7f9fed1/users/0b113168342442298dfba8791620fe9d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:40.459177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=003fa09ccf014b76ad53b88fa7f9fed1, user_id=0b113168342442298dfba8791620fe9d, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:40.464470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:40.496389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1091/2173] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:40 2022] PUT /identity/v3/projects/003fa09ccf014b76ad53b88fa7f9fed1/users/0b113168342442298dfba8791620fe9d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:40.519958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:40.520686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:40.521123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:40.584068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] MFA Rules not processed for user `0b113168342442298dfba8791620fe9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:40.585366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:40.801194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1083/2174] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:40.810874 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:40.871611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:41.065216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91613858-7388-4f9e-bacc-d50f81a8fb9c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:41.071128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:41.072291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:41.072767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:41.073204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:41.074119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:41.076894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:41.113549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1092/2175] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:40 2022] POST /identity/v3/projects => generated 385 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:41.119423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:41.143577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ba87d-9ef2-4192-85c6-d3800b6c91cf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:41.280485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:41.282206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:41.283791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:41.284368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:41.285385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:41.288236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:41.348191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1084/2176] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:41 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:41.353326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:41.371083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc027d2c-952b-4c4a-8c82-5b8a938d0270 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:41.380286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:41.381419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:41.381878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:41.382319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:41.384211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:41.386334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:41.464068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1093/2177] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:41.472642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:41.508073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84326c-2cbd-4811-a8b3-4527cce31bfc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:41.538950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:41.549772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:41.552122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:41.553958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] PATH_INFO: `/v3/projects/0c773b78078e4685a6d8bfac07cd7211/users/2e9678ba9e8b4905a9ba3033c43ec2a2/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:41.617903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c773b78078e4685a6d8bfac07cd7211, user_id=2e9678ba9e8b4905a9ba3033c43ec2a2, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:41.627598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:41.706347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1085/2178] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:41 2022] PUT /identity/v3/projects/0c773b78078e4685a6d8bfac07cd7211/users/2e9678ba9e8b4905a9ba3033c43ec2a2/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:41.708018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:41.736719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57dec6-de90-423f-bf46-f489b4f5a6a4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:41.946971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:41.948138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:41.956014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:41.956661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:41.969287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:41.981855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31d7a15-a199-4398-b2b5-e2275f59a83e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:42.061447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1094/2179] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:42.066443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:42.098702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098adffe-038f-4d67-b7cd-e0f47b1dcc45 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:42.118846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:42.132273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:42.132814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:42.133424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] PATH_INFO: `/v3/projects/0c773b78078e4685a6d8bfac07cd7211/users/2e9678ba9e8b4905a9ba3033c43ec2a2/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:42.137501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c773b78078e4685a6d8bfac07cd7211, user_id=2e9678ba9e8b4905a9ba3033c43ec2a2, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:42.145071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:42.200058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1086/2180] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:42 2022] PUT /identity/v3/projects/0c773b78078e4685a6d8bfac07cd7211/users/2e9678ba9e8b4905a9ba3033c43ec2a2/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:42.211808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:42.212791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:42.213268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:42.329646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] MFA Rules not processed for user `2e9678ba9e8b4905a9ba3033c43ec2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:42.332130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:42.490732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1095/2181] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:42.511388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:42.514664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:42.568399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:42.731510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e76920-e1a8-4b1f-90ff-c2b7bd48c656 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:42.753307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:42.758820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:42.762597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:42.763234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:42.854326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:42.856560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:42.870468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1087/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:45.073603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:45.076729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:45.122004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c432a0-fba1-4f40-b539-b9ab3ad46c78 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:45.132796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:45.133837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:45.134304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:45.134786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:45.162884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:45.308603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:45.310490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:45.352653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1096/2183] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:46.712621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:46.762001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3bd28-b81b-42d5-a2a9-d1f259954313 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:46.774774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:46.775448 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:46.775730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:46.776006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:46.807814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:46.807814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d4bf4-b8ec-46c4-8c75-0052f6dde710 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:46.849872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1088/2184] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:28:51.543328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:51.934198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313855b-2695-4315-b158-f2f0e835ea23 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:51.935203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313855b-2695-4315-b158-f2f0e835ea23 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:51.935456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313855b-2695-4315-b158-f2f0e835ea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:51.937292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6e3c3-34d3-441f-ab6b-50c779ab2082 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:51.951973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:51.954289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:51.958108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:51.958835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] PATH_INFO: `/v3/users/1594c4449e19424e9f0163763bd1a57c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:51.964237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1594c4449e19424e9f0163763bd1a57c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:51.977995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:52.039750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9313855b-2695-4315-b158-f2f0e835ea23 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:52.040853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9313855b-2695-4315-b158-f2f0e835ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:52.135348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1594c4449e19424e9f0163763bd1a57c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:52.139724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1594c4449e19424e9f0163763bd1a57c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:52.140734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1594c4449e19424e9f0163763bd1a57c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:52.156886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1594c4449e19424e9f0163763bd1a57c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:52.175132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17d3bf99-23fd-4a4e-9831-8b1b73cb0c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1594c4449e19424e9f0163763bd1a57c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:52.197621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:52.228613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1097/2185] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:51 2022] DELETE /identity/v3/users/1594c4449e19424e9f0163763bd1a57c => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:52.230356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:52.230649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:52.231312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:52.261116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] MFA Rules not processed for user `effe0200dc584f11923e83ed240138ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:52.265742 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:52.293500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1089/2186] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:52.296067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:52.296840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:52.297131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:52.385898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:52.387203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:52.399621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:52.475063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1098/2187] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:52.475063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:52.475063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:52.542165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1090/2188] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:52.554802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:52.556501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:52.665471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215f6b1-c505-494c-ae46-057f69010003 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:52.679001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:52.679001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:52.679001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:52.679495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:52.761725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836c62-06da-40a5-bf19-b63a4c8a7185 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:52.769770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:52.771469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:52.772360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:52.773580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:52.781325 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:52.783390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:52.793786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:52.794425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1099/2189] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:28:52.795458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:52.811190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1091/2190] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:53.434994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:53.436659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:53.485083 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5abbe1-611f-4edf-ac5d-b41a81f92993 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:53.530148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:53.531264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:53.531608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:53.531894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] PATH_INFO: `/v3/projects/d1134cd659be4d9a9ea186e99239a81b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:53.536565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1134cd659be4d9a9ea186e99239a81b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:53.539298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:53.644448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:53.690170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1134cd659be4d9a9ea186e99239a81b', 'request_id': 'req-b7987776-2b92-4497-a89f-a9e946155567'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:53.708536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1134cd659be4d9a9ea186e99239a81b', 'request_id': 'req-b7987776-2b92-4497-a89f-a9e946155567'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:53.728673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1134cd659be4d9a9ea186e99239a81b', 'request_id': 'req-b7987776-2b92-4497-a89f-a9e946155567'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:53.753490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] The token cache is being invalidate because project d1134cd659be4d9a9ea186e99239a81b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:53.754087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:53.755039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:53.759421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1100/2191] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:53 2022] DELETE /identity/v3/projects/d1134cd659be4d9a9ea186e99239a81b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:53.763608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:53.782424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d255e-8ed3-496c-bcc2-9e4032f966a3 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:53.980421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:53.981803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:53.982251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:53.982698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] PATH_INFO: `/v3/users/0b113168342442298dfba8791620fe9d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:53.996537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b113168342442298dfba8791620fe9d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:54.014813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:54.152958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:54.182498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b113168342442298dfba8791620fe9d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:54.206594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b113168342442298dfba8791620fe9d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:54.244106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b113168342442298dfba8791620fe9d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:54.245631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b113168342442298dfba8791620fe9d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:54.246986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b113168342442298dfba8791620fe9d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:54.271920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1092/2192] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:53 2022] DELETE /identity/v3/users/0b113168342442298dfba8791620fe9d => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:54.814759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:54.817560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:54.847948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7987776-2b92-4497-a89f-a9e946155567 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:55.005476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:55.006646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:55.006940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:55.007233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] PATH_INFO: `/v3/projects/003fa09ccf014b76ad53b88fa7f9fed1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:55.013634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=003fa09ccf014b76ad53b88fa7f9fed1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:55.016651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:55.174852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003fa09ccf014b76ad53b88fa7f9fed1', 'request_id': 'req-b741ddf9-eb01-46ea-9682-ab426504e1ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.223706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003fa09ccf014b76ad53b88fa7f9fed1', 'request_id': 'req-b741ddf9-eb01-46ea-9682-ab426504e1ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.234648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003fa09ccf014b76ad53b88fa7f9fed1', 'request_id': 'req-b741ddf9-eb01-46ea-9682-ab426504e1ef'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.266528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] The token cache is being invalidate because project 003fa09ccf014b76ad53b88fa7f9fed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:55.266528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.270059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b741ddf9-eb01-46ea-9682-ab426504e1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.274105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1101/2193] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:54 2022] DELETE /identity/v3/projects/003fa09ccf014b76ad53b88fa7f9fed1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:55.279814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:55.284290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:55.308282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c255-0b70-4348-ae86-d8493bc67a7f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:55.389057 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:55.390115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:55.390474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:55.390823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] PATH_INFO: `/v3/users/2e9678ba9e8b4905a9ba3033c43ec2a2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:55.392484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9678ba9e8b4905a9ba3033c43ec2a2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:55.394742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:55.462865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9678ba9e8b4905a9ba3033c43ec2a2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.473097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9678ba9e8b4905a9ba3033c43ec2a2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.489343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9678ba9e8b4905a9ba3033c43ec2a2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.490167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9678ba9e8b4905a9ba3033c43ec2a2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.490781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9678ba9e8b4905a9ba3033c43ec2a2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:55.517412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1093/2194] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:28:55 2022] DELETE /identity/v3/users/2e9678ba9e8b4905a9ba3033c43ec2a2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:55.730958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a00a-372d-4943-a5b1-30331089d1ed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:55.731278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a00a-372d-4943-a5b1-30331089d1ed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:55.731518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2a00a-372d-4943-a5b1-30331089d1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:55.764455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bea2a00a-372d-4943-a5b1-30331089d1ed None None] MFA Rules not processed for user `d6682b6bd10a4609b68db0bd02f193b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:55.766007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bea2a00a-372d-4943-a5b1-30331089d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:55.896940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:55.940399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1102/2195] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:28:56.415055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:56.417680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:56.536108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:56.598260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2872-4ae9-41f0-8f4d-8d27ab12d3a2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:56.616150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:56.617411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:56.617951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:56.618331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] PATH_INFO: `/v3/projects/0c773b78078e4685a6d8bfac07cd7211` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:56.620913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c773b78078e4685a6d8bfac07cd7211)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:56.626838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:56.748762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c773b78078e4685a6d8bfac07cd7211', 'request_id': 'req-3274d621-9423-4380-bac5-97a1a562e3ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:56.762862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c773b78078e4685a6d8bfac07cd7211', 'request_id': 'req-3274d621-9423-4380-bac5-97a1a562e3ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:56.785192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c773b78078e4685a6d8bfac07cd7211', 'request_id': 'req-3274d621-9423-4380-bac5-97a1a562e3ec'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:56.824436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] The token cache is being invalidate because project 0c773b78078e4685a6d8bfac07cd7211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:28:56.824740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:56.828814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3274d621-9423-4380-bac5-97a1a562e3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:28:56.834341 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1094/2196] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:28:56 2022] DELETE /identity/v3/projects/0c773b78078e4685a6d8bfac07cd7211 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:28:56.839414 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:56.839701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:56.840205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:56.904023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:56.906562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:57.072170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1103/2197] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:57.077215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.077836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.078334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.134603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:57.139870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:57.167594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1095/2198] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:28:57.172065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.183168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04287ad5-c436-49f8-ba8e-57fc64ec16f7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.190115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.190823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.191137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.191387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.192139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:28:57.192498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.194497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.221227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.229110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1104/2199] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:28:57.233481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2562d1c-dfe7-4fe5-b50d-a108ac1fffa7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.239462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.240171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.240460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.245847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.246937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.258328 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.343790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1096/2200] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:28:57 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:57.348173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.363202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cca3-beac-46c5-9daa-26da8f7193c2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.618586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.620397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.623554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.624007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.624879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.630719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.681126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1105/2201] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:28:57 2022] POST /identity/v3/users => generated 285 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:28:57.685752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.698298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0753ca8-a10f-44c4-9c7b-b09f39858c3f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.705880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.707177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.707676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.710217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.711236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.714509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.735820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.737504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1097/2202] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:28:57.756141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016239f-c2f6-4eb6-b1ae-d821dd24eb8e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.776025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.776025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.776025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.776025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.776944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.776944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453e68-bf5c-4f50-92aa-be07077a269c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.806772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:57.812580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1106/2203] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:28:57.828832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4b44c-ecdb-430c-880b-89ee453df29f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:57.857450 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:57.858445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.865199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.865494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] PATH_INFO: `/v3/projects/dbc44788bc594b0b8bfca1fbf8f51805/users/206ebc4b6a014d04b5b16d7de040dc29/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:57.892463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc44788bc594b0b8bfca1fbf8f51805, user_id=206ebc4b6a014d04b5b16d7de040dc29, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:57.892463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:57.952886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1098/2204] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:28:57 2022] PUT /identity/v3/projects/dbc44788bc594b0b8bfca1fbf8f51805/users/206ebc4b6a014d04b5b16d7de040dc29/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:28:57.980537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d6165-930d-442d-8e58-a29bc79dc928 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:57.980537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d6165-930d-442d-8e58-a29bc79dc928 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:57.980537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d6165-930d-442d-8e58-a29bc79dc928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:58.023413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-487d6165-930d-442d-8e58-a29bc79dc928 None None] MFA Rules not processed for user `206ebc4b6a014d04b5b16d7de040dc29`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:58.024809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-487d6165-930d-442d-8e58-a29bc79dc928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:58.081670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:28:58.133765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1107/2205] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:28:58.147928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:28:58.152479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:28:58.315182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce843e-2660-403b-9e8c-4d267675ab5c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:28:58.331800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:28:58.333007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:58.333470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:58.333927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:58.371265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:28:58.374250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:28:58.401786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1099/2206] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:28:59.794090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402807-b796-43dc-b151-409e2550bec8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:28:59.794730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402807-b796-43dc-b151-409e2550bec8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:28:59.794829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e402807-b796-43dc-b151-409e2550bec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:28:59.835336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0e402807-b796-43dc-b151-409e2550bec8 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:28:59.837401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e402807-b796-43dc-b151-409e2550bec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:00.088635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1108/2207] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:00.141512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:00.144056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:00.177508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:00.410693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784a27-5193-4926-a8b3-3425026800b9 service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:00.437912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:00.440360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:00.441167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:00.441808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:00.469319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:00.472436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ae8d6-2eb6-438b-9098-91fada17f0e7 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:00.506198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1100/2208] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:29:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:29:10.063574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=37 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:10.487627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=29 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:19.639320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:19.640533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:19.641078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:19.643322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1109/2209] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:29:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:29:19.653275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604138-4051-43f1-9b77-958edf5a8ae6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:19.653954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604138-4051-43f1-9b77-958edf5a8ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:19.653954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604138-4051-43f1-9b77-958edf5a8ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:19.837160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-70604138-4051-43f1-9b77-958edf5a8ae6 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:19.848207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-70604138-4051-43f1-9b77-958edf5a8ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:20.119102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1101/2210] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:29:22.595628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:22.599281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:22.907684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836b3bd-e4ab-4fc4-9f6a-fefa101b7cf6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:22.914326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:22.915341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:22.915806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:22.916271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:22.918625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:22.931930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:22.934095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:22.946497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1110/2211] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:27.917938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:27.918568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:27.918568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:27.940090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:27.941486 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:27.949039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1102/2212] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:29:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:27.963050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:27.974703 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fe17c-d60a-4edb-968a-aab256788722 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:27.986120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:27.987037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:27.987328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:27.987607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:27.996857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:27.998533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7c378-5386-4c28-93d5-9fd564099f68 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:28.008989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1111/2213] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:29:28.416370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:28.418380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:28.434688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d1de6-801f-4e44-b6fd-f5d02fd99360 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:28.441049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964812b8-acab-4b43-b675-747136307df1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:28.442490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964812b8-acab-4b43-b675-747136307df1 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.442992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964812b8-acab-4b43-b675-747136307df1 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.443400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964812b8-acab-4b43-b675-747136307df1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.456721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964812b8-acab-4b43-b675-747136307df1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:28.458530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964812b8-acab-4b43-b675-747136307df1 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:28.474547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1103/2214] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:28.583641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.583936 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.584216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.585646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1112/2215] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:29:28.593588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d126fe-ffef-4f29-be04-3c4063c81680 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.593882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d126fe-ffef-4f29-be04-3c4063c81680 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.594153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d126fe-ffef-4f29-be04-3c4063c81680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.622214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-08d126fe-ffef-4f29-be04-3c4063c81680 None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:28.623443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-08d126fe-ffef-4f29-be04-3c4063c81680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:28.682245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1104/2216] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:28.696982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:28.699243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:28.719769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a010d-0964-4cd0-9cab-cf91a12d6f5a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:28.724651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:28.725333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.725592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.725883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.734618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:28.736743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3ba1f-aff8-4262-ab24-a205fd7ecc71 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:28.746013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1113/2217] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:28.838058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27881-0d42-4451-bcd2-d28bde5a35ca None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.838352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27881-0d42-4451-bcd2-d28bde5a35ca None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.838638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27881-0d42-4451-bcd2-d28bde5a35ca None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.840084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1105/2218] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:29:28.846158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3cac1-8756-46f7-a324-c1b2db7201bf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.846446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3cac1-8756-46f7-a324-c1b2db7201bf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.846884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3cac1-8756-46f7-a324-c1b2db7201bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.873000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-36f3cac1-8756-46f7-a324-c1b2db7201bf None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:28.874200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36f3cac1-8756-46f7-a324-c1b2db7201bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:28.885475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1114/2219] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:28.902070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27881-0d42-4451-bcd2-d28bde5a35ca None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:28.915018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27881-0d42-4451-bcd2-d28bde5a35ca None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:28.918870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:28.919569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:28.919812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:28.920086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:28.932410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:28.934432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:28.959397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1106/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:29.134461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:29.135053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:29.135053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:29.159393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:29.161990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:29.179696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1115/2221] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:29:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:29.213170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:29.263706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af669b1-473f-4670-8558-01f17703b69a service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:29.297771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:29.297771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:29.297771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:29.297771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:29.315316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:29.317427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:29.326810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:29.342293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1107/2222] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:29:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:29:32.925956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:32.928657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:33.078837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af29ea-ddb2-474a-837a-6b46bb7491e2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:33.085617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:33.086697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:33.087061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:33.087630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] PATH_INFO: `/v3/users/b9d9cb0073b5452698e549792adf6733` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:33.089654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d9cb0073b5452698e549792adf6733)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:33.092152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:33.148178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:33.168373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d9cb0073b5452698e549792adf6733'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.169096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d9cb0073b5452698e549792adf6733'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.169735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d9cb0073b5452698e549792adf6733'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.180284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d9cb0073b5452698e549792adf6733'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.193295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7fafd53d-dcb0-4d71-b151-fb213621ee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d9cb0073b5452698e549792adf6733'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.217471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1116/2223] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:29:32 2022] DELETE /identity/v3/users/b9d9cb0073b5452698e549792adf6733 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:29:33.730740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:33.745534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df725-c537-4008-84b2-cfe7dda3921e service swift] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:33.819011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:33.820439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:33.820986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:33.821525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] PATH_INFO: `/v3/projects/b99a389f8cd34552ae1b3c30803669ae` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:33.823697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99a389f8cd34552ae1b3c30803669ae)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:33.826334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:33.906067 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99a389f8cd34552ae1b3c30803669ae', 'request_id': 'req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.918621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99a389f8cd34552ae1b3c30803669ae', 'request_id': 'req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.930167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99a389f8cd34552ae1b3c30803669ae', 'request_id': 'req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.945086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] The token cache is being invalidate because project b99a389f8cd34552ae1b3c30803669ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:29:33.945375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.946064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b32ab1ee-af0d-4dd7-a56b-678da26aaaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:33.947665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1108/2224] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:29:33 2022] DELETE /identity/v3/projects/b99a389f8cd34552ae1b3c30803669ae => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:29:33.954750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:33.955660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:33.955660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:33.975710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:33.977476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:34.072275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1117/2225] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:34.081372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.081809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.082264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.118177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:34.119532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:34.130705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1109/2226] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:34.135927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.152172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7954c4-9e74-4ac7-a11a-52a133744301 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.171131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.171851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.172133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.172454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.173336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:29:34.173715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.175501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.193496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1118/2227] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:29:34.197883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.211555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45e49-15f4-4d1b-808f-361d488c3145 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.238425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.239274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.239783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.240162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.241030 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.243155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.285107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1110/2228] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:29:34 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:29:34.287150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.306764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79214e-64af-4074-a0ce-498f8b4ec349 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.411281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.412104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.412618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.412974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.413741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.415850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.441064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1119/2229] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:29:34 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:29:34.446752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.458711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9aed6-e0d0-4d71-86f7-cb3b82e81f66 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.470956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.471673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.471953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.472224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.473089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.474871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.496634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1111/2230] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:29:34.496863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.510286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77040d-3626-4a65-8003-9abfdfdb485d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.516416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.517833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.518121 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.518405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.519521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.521348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65edc85-6b27-49f7-9b73-9ffcb8961623 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.540887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1120/2231] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:29:34.544219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.565980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6fc71-fafc-4580-80ec-d4a5dee10207 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.574446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.576010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.576549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.577036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] PATH_INFO: `/v3/projects/fe9612aa8fdd4be0a4a3ecfaefb96fca/users/4e1f142fbc2a4159b5ad3a672170db10/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.604909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9612aa8fdd4be0a4a3ecfaefb96fca, user_id=4e1f142fbc2a4159b5ad3a672170db10, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.610217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.636250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1112/2232] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:29:34 2022] PUT /identity/v3/projects/fe9612aa8fdd4be0a4a3ecfaefb96fca/users/4e1f142fbc2a4159b5ad3a672170db10/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:29:34.642332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9fc2-6d27-4a51-92e3-bce13ac10aa8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.642647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9fc2-6d27-4a51-92e3-bce13ac10aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.642928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e9fc2-6d27-4a51-92e3-bce13ac10aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.664436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-060e9fc2-6d27-4a51-92e3-bce13ac10aa8 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:34.665725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-060e9fc2-6d27-4a51-92e3-bce13ac10aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:34.677990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:34.764738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1121/2233] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:34.778101 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:34.780149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:34.818120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:34.889142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0e3c6-3969-47d9-9e05-c8a0df5b2e67 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:34.894595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:34.895304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:34.895573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:34.895859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:34.906508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:34.909543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:34.922317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1113/2234] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:36.014762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61465ec-471b-4414-8f42-afac9b350816 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:36.015483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61465ec-471b-4414-8f42-afac9b350816 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:36.015483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61465ec-471b-4414-8f42-afac9b350816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:36.077603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c61465ec-471b-4414-8f42-afac9b350816 None None] MFA Rules not processed for user `4e1f142fbc2a4159b5ad3a672170db10`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:36.081354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c61465ec-471b-4414-8f42-afac9b350816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:36.290370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1122/2235] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:36.306374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:36.310325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:36.562671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3baf-d39d-4f49-860f-932e9019e9ed service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:36.577969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:36.578700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:36.579242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:36.579539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:36.594438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:36.596216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80c9c4-8240-4c43-9689-4fc7f388ac37 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:36.611686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1114/2236] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:41.534609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:41.535458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:41.535458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:41.556282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] MFA Rules not processed for user `4e1f142fbc2a4159b5ad3a672170db10`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:41.557574 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:41.568361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1123/2237] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:43.749748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:43.752205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:43.752794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:43.842585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:43.846199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:43.974449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1115/2238] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:29:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:44.005783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:44.008009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:44.180556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99746917-b43b-4ffe-8d88-c028f6f19ddf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:44.192925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:44.194051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:44.194634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:44.195143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:44.209741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:44.211879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d1ef-3953-423d-8b20-e866f7ad0ccb service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:44.222634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1124/2239] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:29:52.770449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:52.772224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:52.789470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a505a-6e44-49e3-b6fb-a2967efbb3d9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:52.794591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:52.795500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:52.795784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:52.796060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] PATH_INFO: `/v3/users/206ebc4b6a014d04b5b16d7de040dc29` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:52.797425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=206ebc4b6a014d04b5b16d7de040dc29)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:52.799579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:52.821142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:52.871465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206ebc4b6a014d04b5b16d7de040dc29'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:52.880872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206ebc4b6a014d04b5b16d7de040dc29'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:52.895634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206ebc4b6a014d04b5b16d7de040dc29'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:52.896306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206ebc4b6a014d04b5b16d7de040dc29'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:52.897039 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206ebc4b6a014d04b5b16d7de040dc29'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:52.920116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1116/2240] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:29:52 2022] DELETE /identity/v3/users/206ebc4b6a014d04b5b16d7de040dc29 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:29:52.925118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:52.925730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:52.925730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:52.949375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:52.951700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:52.956302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:53.026245 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1125/2241] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:53.042577 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:53.045005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:53.126571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f4b5c-53db-4bda-9182-b99c92c3a86b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:53.136405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:53.137727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:53.138348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:53.138989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:53.157989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:53.160415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10f04-ffbe-43aa-bcbc-281e893574f4 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:53.173414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1117/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:53.700549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:53.702330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:53.723924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4dda9-7c6e-4d03-bf67-75b720300a71 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:53.731485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:53.732578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:53.732961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:53.733330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] PATH_INFO: `/v3/projects/dbc44788bc594b0b8bfca1fbf8f51805` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:53.734684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc44788bc594b0b8bfca1fbf8f51805)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:53.737321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:53.868104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc44788bc594b0b8bfca1fbf8f51805', 'request_id': 'req-548dad90-4f4e-4da8-8d19-ff5b84408f24'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:53.875784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc44788bc594b0b8bfca1fbf8f51805', 'request_id': 'req-548dad90-4f4e-4da8-8d19-ff5b84408f24'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:53.890616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc44788bc594b0b8bfca1fbf8f51805', 'request_id': 'req-548dad90-4f4e-4da8-8d19-ff5b84408f24'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:53.942524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] The token cache is being invalidate because project dbc44788bc594b0b8bfca1fbf8f51805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:29:53.942524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:53.949381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-548dad90-4f4e-4da8-8d19-ff5b84408f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:29:53.950504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1126/2243] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:29:53 2022] DELETE /identity/v3/projects/dbc44788bc594b0b8bfca1fbf8f51805 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:29:53.955835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:53.956733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:53.956733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:53.988425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:53.991584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:54.069926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1118/2244] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:29:54.078691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.078977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.079280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.103681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:54.110604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:54.120935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1127/2245] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:54.126810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.145319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d0341-b47c-46d4-b927-ed9f6649a092 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.150457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.151212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.151491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.151815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.152673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:29:54.153019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.155007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.169827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1119/2246] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:29:54.178331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.196786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c7713-8aaf-41f8-b4cf-7977a09ed83d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.204238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.205517 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.206939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.207576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.209246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.213178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.259027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1128/2247] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:29:54 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:29:54.263198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.274172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6651-71f8-4a1e-a732-c6fd2af161aa admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.299355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:54.347532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.348200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.348653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.348872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.349658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.351821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.376891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1120/2248] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:29:54 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:29:54.381534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.395678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858548e-f6fa-4743-a950-72ca77e00b5b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.400623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.401315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.401605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.401886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.402633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.404305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.416497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1129/2249] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:29:54.421035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.435300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcec55-40d1-48b8-bdcd-9c987c4a0509 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.440935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.441713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.442145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.442430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.443219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.445332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e42c07-25bd-4e65-ba75-2229b7fa2428 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.456883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1121/2250] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:29:54.461179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.472384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a79a11-5ba1-4582-8929-d3b6a20b41af admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.478872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.479875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.480162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.480483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] PATH_INFO: `/v3/projects/64340474d55147bfae2d41d3f671a5e4/users/db2a001f1f6d4ac5a220f035473c631a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.504868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64340474d55147bfae2d41d3f671a5e4, user_id=db2a001f1f6d4ac5a220f035473c631a, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.507615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.531190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1130/2251] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:29:54 2022] PUT /identity/v3/projects/64340474d55147bfae2d41d3f671a5e4/users/db2a001f1f6d4ac5a220f035473c631a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:29:54.604611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f7fb-52e7-495c-a305-4394ea79afd7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.604890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f7fb-52e7-495c-a305-4394ea79afd7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.605171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f7fb-52e7-495c-a305-4394ea79afd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.643358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a127f7fb-52e7-495c-a305-4394ea79afd7 None None] MFA Rules not processed for user `db2a001f1f6d4ac5a220f035473c631a`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:54.645093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a127f7fb-52e7-495c-a305-4394ea79afd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:54.738655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1122/2252] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:29:54.750428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:29:54.752622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:29:54.812040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:29:54.854303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08600-dc62-4047-803f-4ff7638bd328 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:29:54.858047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:29:54.858718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:54.859005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:54.859287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:54.871330 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:29:54.873668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:29:54.892513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1131/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:29:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:29:59.917249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ccb-e102-40ab-acf6-df9f4e82ec32 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:29:59.917789 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ccb-e102-40ab-acf6-df9f4e82ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:29:59.918087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46629ccb-e102-40ab-acf6-df9f4e82ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:29:59.950963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-46629ccb-e102-40ab-acf6-df9f4e82ec32 None None] MFA Rules not processed for user `23abc809ab644c1da0270a35ac8bab21`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:29:59.952398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-46629ccb-e102-40ab-acf6-df9f4e82ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:00.104834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1123/2254] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:29:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:30:04.881680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=13 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:30:06.331162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:30:06.411232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377e76-5ea2-48a3-b368-a3231ecea893 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:30:06.416654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:30:06.417616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:06.417969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:06.418490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] PATH_INFO: `/v3/users/db2a001f1f6d4ac5a220f035473c631a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:06.419957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2a001f1f6d4ac5a220f035473c631a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:30:06.422031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:30:06.484891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2a001f1f6d4ac5a220f035473c631a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:06.485825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2a001f1f6d4ac5a220f035473c631a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:06.486900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2a001f1f6d4ac5a220f035473c631a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:06.496792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2a001f1f6d4ac5a220f035473c631a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:06.506474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2a001f1f6d4ac5a220f035473c631a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:06.528661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1132/2255] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:30:06 2022] DELETE /identity/v3/users/db2a001f1f6d4ac5a220f035473c631a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:30:06.531697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:06.532457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:06.532457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:06.552924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:30:06.554501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:06.612135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:30:06.660163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1124/2256] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:30:06.672078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:30:06.673910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:06.774402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d4b82-128b-4a4e-801c-1b6aca8ee6c4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:30:06.786303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:30:06.787844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:06.788343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:06.788839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:06.807852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:30:06.812307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c0b71-177b-4440-831c-b49e844b51e0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:30:06.831143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1133/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:30:07.342308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:30:07.343770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:07.361903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261846d-8a77-4cec-9aa9-e42c8462497b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:30:07.368232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:30:07.369289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:07.369473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:07.369774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] PATH_INFO: `/v3/projects/64340474d55147bfae2d41d3f671a5e4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:07.371036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64340474d55147bfae2d41d3f671a5e4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:30:07.373142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:30:07.442166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64340474d55147bfae2d41d3f671a5e4', 'request_id': 'req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:07.453061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64340474d55147bfae2d41d3f671a5e4', 'request_id': 'req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:07.466386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64340474d55147bfae2d41d3f671a5e4', 'request_id': 'req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:07.491221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] The token cache is being invalidate because project 64340474d55147bfae2d41d3f671a5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:30:07.491749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:07.493078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:30:07.495032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1125/2258] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:30:07 2022] DELETE /identity/v3/projects/64340474d55147bfae2d41d3f671a5e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:30:10.046884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:10.047415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:10.047518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:10.072649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:30:10.074124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:10.093779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:30:10.137212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1134/2259] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:30:10.151662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:30:10.153028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:10.231494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ecbd0-ed29-44dc-996c-3ac1f62b7dd4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:30:10.237555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:30:10.237947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:10.238234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:10.238516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:10.247748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:30:10.249573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bc000-ccd9-406c-a510-96ac79df5c20 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:30:10.261184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1126/2260] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:30:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:30:12.614850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:30:12.616604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:12.679184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cbc71-90b7-4750-bef0-445f95d61f43 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:30:12.683061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:30:12.684089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:12.684405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:12.684685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:12.686568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:12.750856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:30:12.752349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfd538-d681-4475-9370-7e0c9ead0869 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:30:12.762635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1135/2261] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:30:20.255880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:30:22.757824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=65 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:30:52.004612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:30:52.005288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:30:52.005288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:30:52.026541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] MFA Rules not processed for user `effe0200dc584f11923e83ed240138ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:30:52.027817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:30:52.092977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1127/2262] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:31:02.085339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=25 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:31:07.194666 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:07.195262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:07.195262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:07.213617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] MFA Rules not processed for user `4e1f142fbc2a4159b5ad3a672170db10`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:31:07.214929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:07.277037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1136/2263] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:31:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:31:16.477642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:16.479731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:16.492722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b982b4-0c35-4795-bfc5-4734e2c321d9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:16.496927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:16.497671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:16.498018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:16.498300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:16.500173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:16.581514 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:16.583663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac33e9-585d-42ca-ba32-01583af48e16 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:16.597390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1128/2264] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:17.269392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=25 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:31:18.047770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:18.058533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c5ec1-47a8-4f21-99e2-f50ad092dcfc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:18.062562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:18.063228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:18.063507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:18.063778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:18.065622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:18.081037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:18.082569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da9a28-2e0e-4673-bb7f-53767e7d38ba service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:18.098786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1137/2265] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:22.770049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:22.770588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:22.770701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:22.788544 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] MFA Rules not processed for user `e9dab30db0a44655828384db53df29f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:31:22.789791 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:22.849331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1129/2266] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:31:28.089692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:31:32.841601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=59 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:31:41.876062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:41.876938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:41.878227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:41.897277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] MFA Rules not processed for user `995fa1760f5241588a493e5f894b0e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:31:41.898607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:41.963069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1138/2267] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:31:42.285322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:42.296876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0398-d02f-490c-8b40-50f790c6b390 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:42.301610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:42.302553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:42.302832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:42.303120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:42.305288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:42.317234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:42.318769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:42.328207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1130/2268] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:42.727109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:42.728809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:42.850386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3013af-3af7-4aa7-a331-8ed1233bb829 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:42.865094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:42.865094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:42.865094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:42.865094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:42.866373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:42.881616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:42.884004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:42.913812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1139/2269] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:43.943690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:43.945262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:43.963965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf9cdf-e6c3-4427-a64a-8d43a982ff44 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:43.969977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:43.970714 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:43.971146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:43.971642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:43.973537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:43.984957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:43.987003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:43.998046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1131/2270] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:44.297018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:44.301456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:44.333289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4a75c-4da3-4709-ac1c-6e0ec04304b3 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:44.343857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:44.345314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:44.346500 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:44.347286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:44.350780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:44.367686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:44.369610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76b76e-ac1f-486b-aca1-00100c72cf06 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:44.379831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1140/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:47.350491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:47.352796 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:47.506504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2805b-9f57-4cf1-a103-e60f220cf256 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:47.523710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:47.525216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:47.525916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:47.526532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:47.532639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:47.581800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:47.584396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:47.629138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1132/2272] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:49.469401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:49.470258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:49.470258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:49.531534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] MFA Rules not processed for user `23abc809ab644c1da0270a35ac8bab21`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:31:49.533912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:49.724939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1141/2273] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:31:50.206837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:50.218860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab3703-c8bc-4598-ad8a-5e5260840a43 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:50.223185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:50.223904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:50.224264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:50.224608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:50.226845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:50.240241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:50.242146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:50.254220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1133/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:31:54.244953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:31:54.263446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd5925-074a-4e23-8431-ea4bd997775f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:31:54.271233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:31:54.272400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:31:54.272900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:31:54.273380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:31:54.277755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:31:54.297087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:31:54.300587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:31:54.320102 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1142/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:00.248463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=44 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:04.309998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:08.024949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:08.041141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef000d13-6c63-4d21-abc2-35bed36eaf36 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:08.047501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:08.048300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:08.048701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:08.049240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:08.051825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:08.068630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:08.070677 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:08.086119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1134/2276] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:14.068731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:14.082251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e33d32-1c3a-4882-9443-f854e42394e7 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:14.088274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:14.089422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:14.089692 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:14.090208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:14.092648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:14.107280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:14.109386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:14.122755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1143/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:14.559981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:14.564242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:14.733985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa48ad-9a17-4561-b93b-7a5e03ef2c69 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:14.740591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:14.741526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:14.741990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:14.742422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] PATH_INFO: `/v3/users/4e1f142fbc2a4159b5ad3a672170db10` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:14.744364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e1f142fbc2a4159b5ad3a672170db10)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:14.746990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:14.817242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e1f142fbc2a4159b5ad3a672170db10'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:14.829442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e1f142fbc2a4159b5ad3a672170db10'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:14.855447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e1f142fbc2a4159b5ad3a672170db10'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:14.856489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e1f142fbc2a4159b5ad3a672170db10'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:14.857332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e1f142fbc2a4159b5ad3a672170db10'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:14.889000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1135/2278] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:14 2022] DELETE /identity/v3/users/4e1f142fbc2a4159b5ad3a672170db10 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:15.274745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:15.341665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87103-a0b3-43c8-9fc6-4c5aa53ceff8 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:15.519644 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:15.520486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:15.526743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:15.527349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:15.530685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:15.538890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:15.540425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:15.596239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:15.598681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:15.631943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1144/2279] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:15.811854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a13bd2-fb14-485a-a85b-5387ddb6f9c2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:15.826721 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:15.828420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:15.828720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:15.828999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] PATH_INFO: `/v3/projects/fe9612aa8fdd4be0a4a3ecfaefb96fca` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:15.832345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9612aa8fdd4be0a4a3ecfaefb96fca)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:15.834523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:15.868733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:15.984297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9612aa8fdd4be0a4a3ecfaefb96fca', 'request_id': 'req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:15.999466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9612aa8fdd4be0a4a3ecfaefb96fca', 'request_id': 'req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:16.012023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9612aa8fdd4be0a4a3ecfaefb96fca', 'request_id': 'req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:16.037338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] The token cache is being invalidate because project fe9612aa8fdd4be0a4a3ecfaefb96fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:16.037793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:16.038485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:16.049711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1136/2280] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:15 2022] DELETE /identity/v3/projects/fe9612aa8fdd4be0a4a3ecfaefb96fca => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:17.216958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:17.219238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:17.333089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c2646-956d-48fc-b23e-53e26f21359e service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:17.418429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:17.419404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:17.419851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:17.421283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:17.426284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:17.508146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:17.535163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:17.537127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:17.549665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1145/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:22.059541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:22.061177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:22.079747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe73b7c-15f7-4e0e-9e57-fead6ff25df8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:22.085696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:22.086732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.087017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.087274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] PATH_INFO: `/v3/users/effe0200dc584f11923e83ed240138ff` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.088672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] RBAC: Authorizing `identity:delete_user(user_id=effe0200dc584f11923e83ed240138ff)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:22.090650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:22.152058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'effe0200dc584f11923e83ed240138ff'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.165874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'effe0200dc584f11923e83ed240138ff'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.187694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'effe0200dc584f11923e83ed240138ff'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.188400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'effe0200dc584f11923e83ed240138ff'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.189032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ac87840c-745c-40a7-90ec-8f4a1e00449f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'effe0200dc584f11923e83ed240138ff'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.212076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1137/2282] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:22 2022] DELETE /identity/v3/users/effe0200dc584f11923e83ed240138ff => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:22.420681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:22.437906 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6c190-88af-4a6b-8406-532e53c49dab service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:22.540753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:22.541764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.542067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.542401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] PATH_INFO: `/v3/projects/2832173587a6494eb7f73b3ca0a09a0f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.543723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2832173587a6494eb7f73b3ca0a09a0f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:22.545941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:22.614120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2832173587a6494eb7f73b3ca0a09a0f', 'request_id': 'req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.622279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2832173587a6494eb7f73b3ca0a09a0f', 'request_id': 'req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.629683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2832173587a6494eb7f73b3ca0a09a0f', 'request_id': 'req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.650632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] The token cache is being invalidate because project 2832173587a6494eb7f73b3ca0a09a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:22.651363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.652738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a3b09925-6344-4aa5-a8d8-4f73fe4daee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:22.655027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1146/2283] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:22 2022] DELETE /identity/v3/projects/2832173587a6494eb7f73b3ca0a09a0f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:22.661713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.662240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.662639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.695068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:22.697191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:22.839466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1138/2284] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:22.848215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.848763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.849260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.886933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:22.888376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:22.902789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1147/2285] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:22.908010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:22.919136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7be1fe-2b57-45a3-903c-89bbf73028e7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:22.924782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:22.925952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.926458 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.926956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.928340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:32:22.928916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:22.932035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:22.938728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:22.958834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1139/2286] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:32:22.966852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:22.976860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010bddb-176d-44c8-9c1c-62dd4d9b4484 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:22.983681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:22.984819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:22.985195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:22.985509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:22.986493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:22.989028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.023376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1148/2287] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:22 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:23.036029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.051282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25467a75-9e05-4f5e-9790-af2e27eb4539 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.148311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.149477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.149965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.150456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.151730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.155365 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.239204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1140/2288] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:23 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:23.240589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.252236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9a131-2b0e-4884-8124-28bf47d04e73 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.258683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.259399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.259681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.259955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.260770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.262757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.278996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1149/2289] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:32:23.280538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.295428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803624ef-a85a-4a3a-a592-023c47213e76 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.301738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.302740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.303193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.303672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.304746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.306904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437366-279b-455c-b44b-4b9a7cc15d43 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.320807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1141/2290] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:23.325367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.339876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926410c-663d-4f50-a769-0561a816fd5b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.345501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.347008 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.347565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.347918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] PATH_INFO: `/v3/projects/bc83477c5eb744dc90539f07bb9302c0/users/c78bcad09faa40c6ac8be9108a0b58f5/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.369267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc83477c5eb744dc90539f07bb9302c0, user_id=c78bcad09faa40c6ac8be9108a0b58f5, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.372399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.390875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1150/2291] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:23 2022] PUT /identity/v3/projects/bc83477c5eb744dc90539f07bb9302c0/users/c78bcad09faa40c6ac8be9108a0b58f5/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:23.402439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.402735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.403022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.435697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] MFA Rules not processed for user `c78bcad09faa40c6ac8be9108a0b58f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:23.436904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:23.535217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1142/2292] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:23.540105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.546508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:23.610231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b9fe0-92b3-4021-92e6-c3e7e96305ba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.615470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.616153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.616498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.616772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.617488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.619438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.648046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1151/2293] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:23 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:32:23.652570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.665055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9a48c-0ca1-4a50-bdb4-ceed1d810bb5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.727196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.727874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.728150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.728433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.729530 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.731466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.740977 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:23.756855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1143/2294] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:23 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:23.758182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.769598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43037-3844-4f29-bad9-801589918b32 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.775778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.776503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.776785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.777059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.777810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.779481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.791421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1152/2295] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:23.796012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.806809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66017443-a749-4021-ad59-4e13c63437a9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.812392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.813321 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.813603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.813872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] PATH_INFO: `/v3/projects/4700f663252346378aa9454ecc7c7023/users/2debb2c2ca134557890d78ccc4a5827f/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.831301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4700f663252346378aa9454ecc7c7023, user_id=2debb2c2ca134557890d78ccc4a5827f, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.834056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.849347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1144/2296] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:23 2022] PUT /identity/v3/projects/4700f663252346378aa9454ecc7c7023/users/2debb2c2ca134557890d78ccc4a5827f/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:23.853663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.864741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd6c62-e953-4f4a-8f16-36ed76bdb4c7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.931493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.932210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.932513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.933169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.933919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.935619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e972cb-ba57-4c2c-84c9-8655248327c0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.946937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1153/2297] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:23.951627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:23.963911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd59dd-e215-49de-82d7-75c76eb26d11 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:23.969843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:23.970810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:23.971135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:23.971431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] PATH_INFO: `/v3/projects/4700f663252346378aa9454ecc7c7023/users/2debb2c2ca134557890d78ccc4a5827f/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:23.973726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4700f663252346378aa9454ecc7c7023, user_id=2debb2c2ca134557890d78ccc4a5827f, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:23.976465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25580f-3581-4cf6-8bdb-4f82beefcd9d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:23.989257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1145/2298] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:23 2022] PUT /identity/v3/projects/4700f663252346378aa9454ecc7c7023/users/2debb2c2ca134557890d78ccc4a5827f/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:24.001098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:24.001757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:24.001757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:24.037952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] MFA Rules not processed for user `2debb2c2ca134557890d78ccc4a5827f`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:24.039177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:24.062935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:24.063244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:24.063526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:24.082703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:24.083979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:24.126373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:24.138379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1154/2299] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:24.146036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:24.147606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:24.151497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1146/2300] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:24.215024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:24.216650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:24.221883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d757-cf87-458a-b8d8-454345e1bf45 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:24.227777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:24.229043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:24.229632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:24.230293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:24.249513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:24.252589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:24.269550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1155/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:24.299921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cd9f8-d8aa-4f1a-b8ea-bead73423beb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:24.307062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:24.308270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:24.308624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:24.308912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:24.319882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:24.321586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36be9a-686a-40cf-a338-850e2c2b4584 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:24.332700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1147/2302] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:32:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:32:24.804970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:24.888664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdb2e3-c469-4b4b-bbe8-798a016f4a36 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:24.897115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:24.898673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:24.899243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:24.899781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] PATH_INFO: `/v3/users/c78bcad09faa40c6ac8be9108a0b58f5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:24.902154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78bcad09faa40c6ac8be9108a0b58f5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:24.906043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:24.983895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78bcad09faa40c6ac8be9108a0b58f5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:24.984639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78bcad09faa40c6ac8be9108a0b58f5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:24.985243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78bcad09faa40c6ac8be9108a0b58f5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:24.993591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78bcad09faa40c6ac8be9108a0b58f5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.003560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78bcad09faa40c6ac8be9108a0b58f5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.026392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1156/2303] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:24 2022] DELETE /identity/v3/users/c78bcad09faa40c6ac8be9108a0b58f5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:25.031224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:25.031506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:25.031795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:25.052215 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:25.053632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:25.124214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:25.149987 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1148/2304] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:25.162256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:25.164868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:25.210975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:25.258331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94f93c-14d1-4895-8787-4b37c1701f83 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:25.262377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:25.263035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:25.263313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:25.263586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:25.275629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:25.275629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:25.286748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1157/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:25.670482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:25.672597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:25.708783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6bf2a-08bd-4ce0-9b62-77f2a2803bf0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:25.717156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:25.727536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:25.728298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:25.729167 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] PATH_INFO: `/v3/projects/bc83477c5eb744dc90539f07bb9302c0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:25.731007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc83477c5eb744dc90539f07bb9302c0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:25.733717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:25.800417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc83477c5eb744dc90539f07bb9302c0', 'request_id': 'req-f23351ce-fb6e-4408-a1e6-390e79bac0f5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.808339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc83477c5eb744dc90539f07bb9302c0', 'request_id': 'req-f23351ce-fb6e-4408-a1e6-390e79bac0f5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.818342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc83477c5eb744dc90539f07bb9302c0', 'request_id': 'req-f23351ce-fb6e-4408-a1e6-390e79bac0f5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.830526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] The token cache is being invalidate because project bc83477c5eb744dc90539f07bb9302c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:25.831036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.832028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:25.834381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1149/2306] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:25 2022] DELETE /identity/v3/projects/bc83477c5eb744dc90539f07bb9302c0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:25.838672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:25.866739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a3aa-2cc2-443c-9fb7-7edd25aa42eb service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:25.951101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:25.952160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:25.952567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:25.952953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] PATH_INFO: `/v3/users/2debb2c2ca134557890d78ccc4a5827f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:25.955186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2debb2c2ca134557890d78ccc4a5827f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:25.957388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:26.018320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2debb2c2ca134557890d78ccc4a5827f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.019111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2debb2c2ca134557890d78ccc4a5827f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.020172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2debb2c2ca134557890d78ccc4a5827f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.029319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2debb2c2ca134557890d78ccc4a5827f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.039404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93aaa6a2-d130-40c7-8337-751f2e486325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2debb2c2ca134557890d78ccc4a5827f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.062691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1158/2307] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:25 2022] DELETE /identity/v3/users/2debb2c2ca134557890d78ccc4a5827f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:26.385189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:26.387169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:26.402713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23351ce-fb6e-4408-a1e6-390e79bac0f5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:26.482562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:26.488660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:26.488660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:26.488660 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] PATH_INFO: `/v3/projects/4700f663252346378aa9454ecc7c7023` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:26.490665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4700f663252346378aa9454ecc7c7023)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:26.492859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:26.572038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4700f663252346378aa9454ecc7c7023', 'request_id': 'req-39afad9c-61ee-4c83-93d5-413d149acd6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.579068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4700f663252346378aa9454ecc7c7023', 'request_id': 'req-39afad9c-61ee-4c83-93d5-413d149acd6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.592808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4700f663252346378aa9454ecc7c7023', 'request_id': 'req-39afad9c-61ee-4c83-93d5-413d149acd6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.611428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] The token cache is being invalidate because project 4700f663252346378aa9454ecc7c7023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:26.611896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.612797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-39afad9c-61ee-4c83-93d5-413d149acd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:26.615489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1150/2308] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:26 2022] DELETE /identity/v3/projects/4700f663252346378aa9454ecc7c7023 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:32:26.621274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:26.622054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:26.622524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:26.645565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:26.646614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:26.725521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:26.761636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1159/2309] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:32:26.774641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:26.775146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:26.775886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:26.788779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:26.811667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:26.814763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:26.829400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1151/2310] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:26.835914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:26.849998 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c963-6cfd-41d7-aff7-bd64d8308551 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:26.858370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:26.860668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:26.861513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:26.862498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:26.864232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:32:26.865085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:26.868708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:26.889777 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1160/2311] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:32:26.895713 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:26.912879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df3484-c74b-4630-84e9-b7b2accb6785 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:26.921350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:26.922614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:26.924086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:26.924667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:26.926017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:26.929665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:26.975332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1152/2312] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:26 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:26.977443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:26.996839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f1c79-5589-4a5f-b43d-0c7a84e40e4d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.103781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.104931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.105443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.105961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.107317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.111436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.142184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1161/2313] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:26 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:27.148452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.171759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d75bb-16d7-47aa-b06d-5771651984f8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.186906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.188126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.188683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.189251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.190708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.193146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.208821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1153/2314] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:27.214162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.228159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000140e-2faf-49f0-aae3-265bb7520c85 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.237162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.238210 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.239350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.239350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.240172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.242984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e4a33-3c99-4202-aef5-7c505a194ee4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.259862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1162/2315] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:27.265069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.280180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df400c-a02f-4872-af29-c9fca23e8d8e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.286050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.287078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.287427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.288248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] PATH_INFO: `/v3/projects/d35fa3c84ea944118d33de0e580800b4/users/c88143c45dcf43b188b9edc2a00b0ea2/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.322186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35fa3c84ea944118d33de0e580800b4, user_id=c88143c45dcf43b188b9edc2a00b0ea2, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.329758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.364457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1154/2316] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:27 2022] PUT /identity/v3/projects/d35fa3c84ea944118d33de0e580800b4/users/c88143c45dcf43b188b9edc2a00b0ea2/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:32:27.373796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.374341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.374850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.438976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] MFA Rules not processed for user `c88143c45dcf43b188b9edc2a00b0ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:27.441865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:27.547514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1163/2317] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:27.553296 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.661685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ed6-84ba-421a-8e76-a3037bf4d7b7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.667336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.668445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.668889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.669162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.670398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.672018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.704135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1155/2318] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:27 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:27.709010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.726194 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f336e4d-00e1-4696-8985-6f68abbd4bd0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.802534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:27.853371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.854862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.855233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.855474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.856206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.858350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.906949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1164/2319] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:27 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:27.908706 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.927541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861ce46-8d89-4598-908d-f3aa6fd2428a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.938593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.939555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.940312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.940787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.941937 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:27.943740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:27.960828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1156/2320] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:32:27.962994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:27.982963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b2d13-c2c3-4f92-ab5d-285f044ac8d0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:27.993390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:27.994577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:27.995074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:27.995563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:27.997043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.000599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18697ec8-4c78-475c-825f-0dff27570685 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.023014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1165/2321] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:28.029596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.059829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de85cfd-b642-41ec-9d22-0a63ff011137 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.068054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.069745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.070243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.070731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] PATH_INFO: `/v3/projects/c34b07a60fda40f78f082a1af6de71d8/users/3c891a8ce2f34e62b4cf2edf90b6deee/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.091120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34b07a60fda40f78f082a1af6de71d8, user_id=3c891a8ce2f34e62b4cf2edf90b6deee, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.100622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.126484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1157/2322] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:28 2022] PUT /identity/v3/projects/c34b07a60fda40f78f082a1af6de71d8/users/3c891a8ce2f34e62b4cf2edf90b6deee/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:28.140549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.140938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.141417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.219915 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] MFA Rules not processed for user `3c891a8ce2f34e62b4cf2edf90b6deee`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:28.223218 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:28.325176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1166/2323] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:28.330257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.390005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:28.403527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681cc43-f473-4213-a8c7-c8bdcc62b680 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.409569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.410209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.410616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.410890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.411699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.413913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.445517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1158/2324] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:28 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:32:28.449816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.462297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e01596-439a-4487-950d-644ce786cab1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.558250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.558935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.559215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.559488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.560215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.562236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.590152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1167/2325] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:28 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:32:28.594502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.606896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52f1a-13fc-422d-9e61-d06f0349b467 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.612669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.613557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.613945 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.614329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.615230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.617163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.631711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1159/2326] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:28.637105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.659864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5f3b-85fe-4409-8399-cc141b311ff7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.668266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.670015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.670499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.670982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] PATH_INFO: `/v3/projects/b4b9c43abac34ba89530b9ed3768810c/users/f60e1e5686b14c39a68f18e635c0ee64/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.693864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b9c43abac34ba89530b9ed3768810c, user_id=f60e1e5686b14c39a68f18e635c0ee64, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.697368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.717596 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1168/2327] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:28 2022] PUT /identity/v3/projects/b4b9c43abac34ba89530b9ed3768810c/users/f60e1e5686b14c39a68f18e635c0ee64/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:28.722511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.734177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4c245-a0d6-477d-8066-e52e8fe43ca4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.823284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.824382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.825032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.825249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.826105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.827873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e2ed4-4a30-44fd-a853-52a5ad1e63bc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.856820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1160/2328] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:28.858070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:28.877230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f991-6862-461f-897e-da7ce20f0d93 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:28.886856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:28.888879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.889166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.889451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] PATH_INFO: `/v3/projects/b4b9c43abac34ba89530b9ed3768810c/users/f60e1e5686b14c39a68f18e635c0ee64/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:28.892275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b9c43abac34ba89530b9ed3768810c, user_id=f60e1e5686b14c39a68f18e635c0ee64, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:28.895830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:28.903803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:28.915555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1169/2329] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:28 2022] PUT /identity/v3/projects/b4b9c43abac34ba89530b9ed3768810c/users/f60e1e5686b14c39a68f18e635c0ee64/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:32:28.942526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:28.943128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:28.943281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:29.009758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] MFA Rules not processed for user `f60e1e5686b14c39a68f18e635c0ee64`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:29.011271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:29.144382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1161/2330] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:29.156052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:29.158222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:29.266607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c043c15-871f-4c20-9840-f2a45d78edd4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:29.271507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:29.272195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:29.272513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:29.272792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:29.285753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:29.287329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:29.300021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1170/2331] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:30.112358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:30.114274 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:30.135801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ad307-06d8-4178-96ce-987606ad508b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:30.142052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:30.143383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:30.143893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:30.144422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:30.220609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:30.240353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:30.242121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:30.257665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1162/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:34.122015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:34.123536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:34.217573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fabba-23f3-4945-940d-6146d538263d service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:34.223349 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:34.224443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:34.224811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:34.225192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] PATH_INFO: `/v3/users/d6682b6bd10a4609b68db0bd02f193b6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:34.226609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6682b6bd10a4609b68db0bd02f193b6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:34.228741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:34.287576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6682b6bd10a4609b68db0bd02f193b6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.288169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6682b6bd10a4609b68db0bd02f193b6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.288822 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6682b6bd10a4609b68db0bd02f193b6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.297201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6682b6bd10a4609b68db0bd02f193b6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.306491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6682b6bd10a4609b68db0bd02f193b6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.331937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1171/2333] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:34 2022] DELETE /identity/v3/users/d6682b6bd10a4609b68db0bd02f193b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:34.546667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:34.566799 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aa8f5-9783-4127-a6bf-dd79996205be service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:34.653775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:34.655338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:34.655833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:34.656373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] PATH_INFO: `/v3/projects/52c96385e67941648eee6bf532b3f141` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:34.658823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c96385e67941648eee6bf532b3f141)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:34.663051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:34.734304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c96385e67941648eee6bf532b3f141', 'request_id': 'req-98fbf288-08cb-4b63-8f9e-823f369aa249'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.743916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c96385e67941648eee6bf532b3f141', 'request_id': 'req-98fbf288-08cb-4b63-8f9e-823f369aa249'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.753495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c96385e67941648eee6bf532b3f141', 'request_id': 'req-98fbf288-08cb-4b63-8f9e-823f369aa249'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.773505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] The token cache is being invalidate because project 52c96385e67941648eee6bf532b3f141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:34.774007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.775239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.778168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1163/2334] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:34 2022] DELETE /identity/v3/projects/52c96385e67941648eee6bf532b3f141 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:34.784381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:34.786046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:34.802029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba641-9b04-410f-8b44-a1f660e1741c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:34.832457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:34.834239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:34.837582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:34.899432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:34.900212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:34.900535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:34.900809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] PATH_INFO: `/v3/users/c8a163cbe29d49aeb63b4cfa8186fe8e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:34.902189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a163cbe29d49aeb63b4cfa8186fe8e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:34.904219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:34.924592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbf288-08cb-4b63-8f9e-823f369aa249 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:34.929557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:34.930229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:34.930686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:34.930970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:34.933612 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:34.969966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a163cbe29d49aeb63b4cfa8186fe8e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.971068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a163cbe29d49aeb63b4cfa8186fe8e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.972048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a163cbe29d49aeb63b4cfa8186fe8e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:34.979227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:34.987355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a163cbe29d49aeb63b4cfa8186fe8e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.000241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a163cbe29d49aeb63b4cfa8186fe8e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.022745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:35.025208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:35.033660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1172/2335] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:34 2022] DELETE /identity/v3/users/c8a163cbe29d49aeb63b4cfa8186fe8e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:35.044301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1164/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:35.337576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:35.340352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:35.483790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec4612-bbfc-4732-a2b4-5af22d43caca admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:35.492398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:35.493903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:35.494372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:35.494830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] PATH_INFO: `/v3/projects/adf8b075ad134044b7c840b7c53e41e6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:35.497768 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf8b075ad134044b7c840b7c53e41e6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:35.502324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:35.590538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:35.598096 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf8b075ad134044b7c840b7c53e41e6', 'request_id': 'req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.607672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf8b075ad134044b7c840b7c53e41e6', 'request_id': 'req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.618099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf8b075ad134044b7c840b7c53e41e6', 'request_id': 'req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.636079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] The token cache is being invalidate because project adf8b075ad134044b7c840b7c53e41e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:35.636737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.637622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5a2d4e5b-9542-44d3-9d9b-f5dc88bd6843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:35.639677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1173/2337] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:35 2022] DELETE /identity/v3/projects/adf8b075ad134044b7c840b7c53e41e6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:37.717523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:37.719497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:37.736390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8a984-d773-4f25-9174-0905e49ad845 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:37.805697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:37.806621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:37.806904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:37.807180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] PATH_INFO: `/v3/users/c88143c45dcf43b188b9edc2a00b0ea2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:37.808923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88143c45dcf43b188b9edc2a00b0ea2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:37.810926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:37.870780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88143c45dcf43b188b9edc2a00b0ea2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:37.879548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88143c45dcf43b188b9edc2a00b0ea2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:37.894339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88143c45dcf43b188b9edc2a00b0ea2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:37.895125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88143c45dcf43b188b9edc2a00b0ea2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:37.895754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88143c45dcf43b188b9edc2a00b0ea2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:37.915086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1165/2338] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:37 2022] DELETE /identity/v3/users/c88143c45dcf43b188b9edc2a00b0ea2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:37.920002 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:37.920628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:37.920628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:37.940772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:37.942034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:38.019856 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1174/2339] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:32:38.030715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:38.032542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:38.107624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:38.114833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169538ec-56cf-419d-be69-174657bb0b11 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:38.123765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:38.125046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:38.125880 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:38.126621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:38.144462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:38.147538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:38.169231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1166/2340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:38.592253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:38.594019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:38.611320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ea24e-4aaa-4aaa-b3a6-ba321d3b26bc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:38.616541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:38.617469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:38.617750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:38.618023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] PATH_INFO: `/v3/projects/d35fa3c84ea944118d33de0e580800b4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:38.619337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35fa3c84ea944118d33de0e580800b4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:38.621616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:38.694217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35fa3c84ea944118d33de0e580800b4', 'request_id': 'req-595f8ae5-8301-46b8-9c06-c7abe645f45b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.701189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35fa3c84ea944118d33de0e580800b4', 'request_id': 'req-595f8ae5-8301-46b8-9c06-c7abe645f45b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.709089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35fa3c84ea944118d33de0e580800b4', 'request_id': 'req-595f8ae5-8301-46b8-9c06-c7abe645f45b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.723896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] The token cache is being invalidate because project d35fa3c84ea944118d33de0e580800b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:38.724223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.724939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.726619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1175/2341] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:38 2022] DELETE /identity/v3/projects/d35fa3c84ea944118d33de0e580800b4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:38.731364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:38.742406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11542c-1e4f-4803-ae6c-a0bcd6833d7d service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:38.826199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:38.827757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:38.828255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:38.828807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] PATH_INFO: `/v3/users/3c891a8ce2f34e62b4cf2edf90b6deee` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:38.831267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c891a8ce2f34e62b4cf2edf90b6deee)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:38.835447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:38.900781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c891a8ce2f34e62b4cf2edf90b6deee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.912545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c891a8ce2f34e62b4cf2edf90b6deee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.928149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c891a8ce2f34e62b4cf2edf90b6deee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.928961 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c891a8ce2f34e62b4cf2edf90b6deee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.929586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c891a8ce2f34e62b4cf2edf90b6deee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:38.946129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1167/2342] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:38 2022] DELETE /identity/v3/users/3c891a8ce2f34e62b4cf2edf90b6deee => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:39.342183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:39.347287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:39.373669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8ae5-8301-46b8-9c06-c7abe645f45b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:39.463787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:39.474297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:39.475488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:39.475786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:39.476071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] PATH_INFO: `/v3/projects/c34b07a60fda40f78f082a1af6de71d8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:39.478240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34b07a60fda40f78f082a1af6de71d8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:39.481851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:39.545988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34b07a60fda40f78f082a1af6de71d8', 'request_id': 'req-9332422b-68e8-40fd-a996-4feebe1f96d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.556772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34b07a60fda40f78f082a1af6de71d8', 'request_id': 'req-9332422b-68e8-40fd-a996-4feebe1f96d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.571996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34b07a60fda40f78f082a1af6de71d8', 'request_id': 'req-9332422b-68e8-40fd-a996-4feebe1f96d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.583461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] The token cache is being invalidate because project c34b07a60fda40f78f082a1af6de71d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:39.583821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.584582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.586302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1176/2343] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:39 2022] DELETE /identity/v3/projects/c34b07a60fda40f78f082a1af6de71d8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:39.591104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:39.592938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:39.606523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707ecb4-c9a7-46e9-999d-d22cf9d58569 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:39.670286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:39.671100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:39.671396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:39.671685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] PATH_INFO: `/v3/users/f60e1e5686b14c39a68f18e635c0ee64` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:39.673002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f60e1e5686b14c39a68f18e635c0ee64)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:39.675069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:39.691656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:39.731696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f60e1e5686b14c39a68f18e635c0ee64'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.742032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f60e1e5686b14c39a68f18e635c0ee64'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.756882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f60e1e5686b14c39a68f18e635c0ee64'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.757669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f60e1e5686b14c39a68f18e635c0ee64'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.758295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eea2a5b1-8db6-4c1d-9ebc-4d860a5db306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f60e1e5686b14c39a68f18e635c0ee64'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:39.774351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1168/2344] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:39 2022] DELETE /identity/v3/users/f60e1e5686b14c39a68f18e635c0ee64 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:40.091563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.093235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:40.108778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332422b-68e8-40fd-a996-4feebe1f96d3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.182496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.183800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.184086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.184392 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] PATH_INFO: `/v3/projects/b4b9c43abac34ba89530b9ed3768810c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.185749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b9c43abac34ba89530b9ed3768810c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.187795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.241897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b9c43abac34ba89530b9ed3768810c', 'request_id': 'req-4db0df16-7690-451d-aef9-a4bf93f145ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:40.248430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b9c43abac34ba89530b9ed3768810c', 'request_id': 'req-4db0df16-7690-451d-aef9-a4bf93f145ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:40.256952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b9c43abac34ba89530b9ed3768810c', 'request_id': 'req-4db0df16-7690-451d-aef9-a4bf93f145ce'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:40.272621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] The token cache is being invalidate because project b4b9c43abac34ba89530b9ed3768810c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:40.272909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:40.273577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4db0df16-7690-451d-aef9-a4bf93f145ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:40.275192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1177/2345] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:40 2022] DELETE /identity/v3/projects/b4b9c43abac34ba89530b9ed3768810c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:40.281653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.281944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.282224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.309850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:40.311828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:40.386931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1169/2346] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:40.402603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.403172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.403172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.425402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:40.426699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:40.435839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1178/2347] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:40.440866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.453500 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1d56d-1a4d-406b-ae9e-0f96017a1be6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.460906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.461596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.461878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.462156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.462949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:32:40.463258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.465010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.479609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1170/2348] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:32:40.484504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.495638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ce22-9c62-417b-ae96-75ad34371d6f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.501760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.502468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.502748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.503137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.503901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.507065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.512456 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:40.560773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1179/2349] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/projects => generated 409 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:40.568773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.585584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3496043-779e-4b57-99bc-fc40dc793600 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.663785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.664538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.664835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.665118 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.665830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.667845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.692392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1171/2350] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:40.697724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.717207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0728ea7-1aab-41b4-8e97-44947154b049 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.726044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.726537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.727117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.727970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.729470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.732809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.754206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1180/2351] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:40.758180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.769559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422d65f-f7c4-4ca6-8710-9dbb8b8f14d8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.774394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.775079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.775548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.775833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.776681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.778312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709e2f-c86b-4a82-b662-5085e9e0fa4c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.779974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:40.792119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1172/2352] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:40.796635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:40.807224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b9858-4965-47f9-bbec-07f91aba6035 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:40.812280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:40.813206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.813481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.813754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] PATH_INFO: `/v3/projects/1cc823143e9843c4a57f6e8e73777f45/users/1993f22536964b40bdd529b47d35e156/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.830109 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc823143e9843c4a57f6e8e73777f45, user_id=1993f22536964b40bdd529b47d35e156, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:40.832884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:40.847837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1181/2353] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:40 2022] PUT /identity/v3/projects/1cc823143e9843c4a57f6e8e73777f45/users/1993f22536964b40bdd529b47d35e156/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:40.860768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54d23-de26-401a-b3d9-467642461890 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:40.861257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54d23-de26-401a-b3d9-467642461890 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:40.861685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a54d23-de26-401a-b3d9-467642461890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:40.894338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-73a54d23-de26-401a-b3d9-467642461890 None None] MFA Rules not processed for user `1993f22536964b40bdd529b47d35e156`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:40.895617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-73a54d23-de26-401a-b3d9-467642461890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:40.970007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1173/2354] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:32:40.975111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.041719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d850fc-2d19-46d4-bd0e-c77503d1b7d9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.048311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.049498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.049889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.050316 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.051312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.053774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.086790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1182/2355] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:40 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:41.094112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54d23-de26-401a-b3d9-467642461890 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.110579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a54d23-de26-401a-b3d9-467642461890 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.173099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.174120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.174434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.174835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.175785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.177891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.203434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1174/2356] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:41 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:41.208430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.219634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99437986-8033-496c-9b29-f5d913a8e024 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.225064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.225652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.225934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.226209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.226978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.228639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.241736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1183/2357] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:41.246542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.258012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1402f-eca1-4dac-8538-32551c34fe64 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.263086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.263994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.264273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.264587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] PATH_INFO: `/v3/projects/51dba12070534877a087de0d85f35c33/users/0e45b82e2f134c3a8e083654b5ec6f38/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.282095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dba12070534877a087de0d85f35c33, user_id=0e45b82e2f134c3a8e083654b5ec6f38, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.284899 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.301100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1175/2358] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:41 2022] PUT /identity/v3/projects/51dba12070534877a087de0d85f35c33/users/0e45b82e2f134c3a8e083654b5ec6f38/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:41.305786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.318173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f09bf9-9a1d-4062-9fdf-74b4db774ff5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.386863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.387643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.387965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.388333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.389118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.390897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c0707-1995-45e4-b1e7-2c6159e29f9e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.404164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1184/2359] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:41.415905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.420473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716cb3e-3652-4273-b346-8959cd108544 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.426630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.427566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.427868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.428141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] PATH_INFO: `/v3/projects/51dba12070534877a087de0d85f35c33/users/0e45b82e2f134c3a8e083654b5ec6f38/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.430631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51dba12070534877a087de0d85f35c33, user_id=0e45b82e2f134c3a8e083654b5ec6f38, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.433812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.445970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1176/2360] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:41 2022] PUT /identity/v3/projects/51dba12070534877a087de0d85f35c33/users/0e45b82e2f134c3a8e083654b5ec6f38/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:41.457113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.457408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.457690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.483966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:41.488212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] MFA Rules not processed for user `0e45b82e2f134c3a8e083654b5ec6f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:41.490511 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:41.573617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1185/2361] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:32:41.586011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.588027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:41.638593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:41.660633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe54b97-c6b5-45ac-8fb7-7547cf7e973f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.665525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.666353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.666682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.667070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.680040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.681815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ba99-d353-47db-a12d-328d6cbc2ea7 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:41.697991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1177/2362] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:41.877283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:41.887689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e9d43-8020-4f0a-a335-ff3a3d4eadc0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:41.893293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:41.894328 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:41.894617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:41.894901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:41.997177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:41.998668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:42.009160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1186/2363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:47.890160 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:47.890701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:47.890701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:47.915705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:47.917267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:47.999761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1178/2364] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:48.021167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:48.022905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:48.125442 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e982b-60cb-4f51-9e94-5e35ba87c90c service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:48.135698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:48.136712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:48.137063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:48.137430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:48.153422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:48.156656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:48.189454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1187/2365] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:32:48.829004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:48.831191 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:49.034874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caae9cf-7833-464e-8a52-9bada7853367 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:49.047469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:49.048273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:49.048764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:49.049049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:49.060245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:49.085193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:49.095605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a689c2-bcb3-433f-9818-ee5739d47d85 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:49.101664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1179/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:51.016882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:51.080881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:51.162879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70663c-510e-4f12-b139-55517bc56fcd service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:51.171340 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:51.172976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:51.176763 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:51.179711 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] PATH_INFO: `/v3/users/1993f22536964b40bdd529b47d35e156` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:51.181289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1993f22536964b40bdd529b47d35e156)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:51.184127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:51.268432 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1993f22536964b40bdd529b47d35e156'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:51.268917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1993f22536964b40bdd529b47d35e156'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:51.273122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1993f22536964b40bdd529b47d35e156'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:51.282489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1993f22536964b40bdd529b47d35e156'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:51.293459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1993f22536964b40bdd529b47d35e156'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:51.323204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1188/2367] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:51 2022] DELETE /identity/v3/users/1993f22536964b40bdd529b47d35e156 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:32:51.328212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:51.328907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:51.328907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:51.354113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:51.355400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:51.488158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1180/2368] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:51.497566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:51.499931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:51.610254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc32fa-d4c5-4be9-b10b-6fd891c3304c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:51.614791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:51.615474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:51.615760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:51.616037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:51.628235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:51.629850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:51.643386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1189/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:32:52.298107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:52.300924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:52.327543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f46b42-f9b6-479c-a5a6-5e6c59f4db23 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:52.338986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:52.340889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:52.342401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:52.342975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] PATH_INFO: `/v3/projects/1cc823143e9843c4a57f6e8e73777f45` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:52.345772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc823143e9843c4a57f6e8e73777f45)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:52.350068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:52.363358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:52.443603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc823143e9843c4a57f6e8e73777f45', 'request_id': 'req-4343d37d-281d-4ae0-a86d-124bc0aa6958'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.452344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc823143e9843c4a57f6e8e73777f45', 'request_id': 'req-4343d37d-281d-4ae0-a86d-124bc0aa6958'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.465109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc823143e9843c4a57f6e8e73777f45', 'request_id': 'req-4343d37d-281d-4ae0-a86d-124bc0aa6958'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.480434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] The token cache is being invalidate because project 1cc823143e9843c4a57f6e8e73777f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:52.480828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.484659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.488412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1181/2370] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:52 2022] DELETE /identity/v3/projects/1cc823143e9843c4a57f6e8e73777f45 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:52.492093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:52.512455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c475d3a-297f-411c-8d19-b434efcbf0ab service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:52.584387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:52.613261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:52.614343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:52.614699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:52.615060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] PATH_INFO: `/v3/users/0e45b82e2f134c3a8e083654b5ec6f38` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:52.616490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e45b82e2f134c3a8e083654b5ec6f38)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:52.618573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:52.690045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e45b82e2f134c3a8e083654b5ec6f38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.694490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e45b82e2f134c3a8e083654b5ec6f38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.694913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e45b82e2f134c3a8e083654b5ec6f38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.708139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e45b82e2f134c3a8e083654b5ec6f38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.718601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-32457683-f8fc-4477-80e1-0e4b65a15a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e45b82e2f134c3a8e083654b5ec6f38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:52.741393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1190/2371] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:32:52 2022] DELETE /identity/v3/users/0e45b82e2f134c3a8e083654b5ec6f38 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:53.247234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:53.249403 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:53.302059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343d37d-281d-4ae0-a86d-124bc0aa6958 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:53.393053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:53.394494 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.394886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.395350 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] PATH_INFO: `/v3/projects/51dba12070534877a087de0d85f35c33` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:53.397317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51dba12070534877a087de0d85f35c33)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:53.400972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:53.482618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51dba12070534877a087de0d85f35c33', 'request_id': 'req-d0166ea7-f503-469e-b122-9933892bf795'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:53.492916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51dba12070534877a087de0d85f35c33', 'request_id': 'req-d0166ea7-f503-469e-b122-9933892bf795'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:53.504262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51dba12070534877a087de0d85f35c33', 'request_id': 'req-d0166ea7-f503-469e-b122-9933892bf795'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:53.518004 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] The token cache is being invalidate because project 51dba12070534877a087de0d85f35c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:32:53.518316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:53.518992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0166ea7-f503-469e-b122-9933892bf795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:32:53.520769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1182/2372] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:32:53 2022] DELETE /identity/v3/projects/51dba12070534877a087de0d85f35c33 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:53.526219 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.527232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.527665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:53.551784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:53.553556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:53.668603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1191/2373] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:53.677559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.677559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.677559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:53.699002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:53.700645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:53.710278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1183/2374] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:53.715576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:53.726843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc606a9-9a69-4c66-bc13-abdd63594a27 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:53.733524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:53.734399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.734788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.735139 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:53.735963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:32:53.736421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:53.738128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:53.764638 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1192/2375] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:32:53.776645 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:53.797655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90200753-d91a-4d8f-a9ca-66b4dbe14c78 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:53.806348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:53.807901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.808224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.808580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:53.809818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:53.812163 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:53.842547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:53.879477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1184/2376] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:53 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:53.883613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:53.896940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fafd99-d1b3-49cc-a2e0-6c8e5df25bdc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:53.975189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:53.990947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:53.991269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:53.991827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:53.992800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.003715 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.006523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.059809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1193/2377] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:53 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:54.061006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.086021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249d42-663f-478b-93f3-ad7a76a45be0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.092739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.093434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.093725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.094275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.095717 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.101352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.122338 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1185/2378] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:54.130880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.147120 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203364-000f-4189-8536-7e2760971e46 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.152817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.153743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.154111 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.154463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.155297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.157072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d12f6-fd44-4023-8281-1df7cfbc3556 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.170909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1194/2379] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:32:54.176131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.190737 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481191dd-4527-4376-b1c2-a00fe2d6460f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.196697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.197618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.197906 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.198176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] PATH_INFO: `/v3/projects/105d46d520cb4a2191dda7258a2c8916/users/459daebc69b64533900752ba4cac233d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.216410 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=105d46d520cb4a2191dda7258a2c8916, user_id=459daebc69b64533900752ba4cac233d, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.220196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.253008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1186/2380] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:54 2022] PUT /identity/v3/projects/105d46d520cb4a2191dda7258a2c8916/users/459daebc69b64533900752ba4cac233d/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:54.264627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.265363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.266335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.315250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] MFA Rules not processed for user `459daebc69b64533900752ba4cac233d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:54.316764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:54.417883 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1195/2381] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:32:54.423853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.534263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e1ab-c349-4be9-b525-a66cb2af29ed admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.542190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.543074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.543364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.544033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.576655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.576655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.616915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1187/2382] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:32:54 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:54.623368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.639251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7b944-4d66-45f3-bebb-c89e55fa3f62 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.748774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.749668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.750054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.750390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.751208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.753793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.788709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1196/2383] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:32:54 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:32:54.794960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.814540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ae5f2-8563-4611-a3cd-6d1435cc497f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.823055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.824931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.825827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.826228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.827060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.828624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.857497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1188/2384] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:32:54.867282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.884064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b366e3-8cbc-4074-acd7-0098b07098d3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:54.892459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:54.895146 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:54.895443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:54.895717 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] PATH_INFO: `/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6/users/d54136a637ad475fb52bdac5a9c981c9/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:54.916285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5064a8ee8a04a5196fcadabe4a20bc6, user_id=d54136a637ad475fb52bdac5a9c981c9, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:54.919129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:54.946438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1197/2385] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:54 2022] PUT /identity/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6/users/d54136a637ad475fb52bdac5a9c981c9/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:32:54.950548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:54.970171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a213ea9-5d41-46c7-bf1a-44724442524d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:55.080651 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:55.082189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:55.083280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:55.083792 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:55.086258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:55.091102 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a4b11-dfb3-4bc7-b3d7-52cf88a18572 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:55.112878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1189/2386] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:32:55.122943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:55.145332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72469fe7-3bea-438a-ab5b-6907020ead66 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:55.156220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:55.157793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:55.158277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:55.159055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] PATH_INFO: `/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6/users/d54136a637ad475fb52bdac5a9c981c9/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:55.161551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5064a8ee8a04a5196fcadabe4a20bc6, user_id=d54136a637ad475fb52bdac5a9c981c9, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:55.165061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:55.178292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1198/2387] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:32:55 2022] PUT /identity/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6/users/d54136a637ad475fb52bdac5a9c981c9/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:32:55.190738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:55.191056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:55.191318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:55.223580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] MFA Rules not processed for user `d54136a637ad475fb52bdac5a9c981c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:32:55.225147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:55.252404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:55.351635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1190/2388] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:32:55.366013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:55.368110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:32:55.400494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:32:55.466775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce295f-4bb9-4023-9f65-1f9993bc1aeb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:55.472807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:55.473582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:55.473930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:55.474521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:55.489668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:55.492610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:55.512904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1199/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:32:55.638116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:32:55.657219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da474f8b-fd7b-4beb-a3b0-11c3bebf9491 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:32:55.674347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:32:55.675071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:32:55.675415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:32:55.675696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:32:55.760824 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:32:55.763952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261c6c3-7563-44aa-ab73-2c045b1f029e service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:32:55.783885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1191/2390] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:32:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:33:01.125171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:01.216828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404620a-7754-488b-a72c-4c096c1bc6c1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:01.225797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:01.227058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:01.227437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:01.228283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] PATH_INFO: `/v3/users/459daebc69b64533900752ba4cac233d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:01.230311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459daebc69b64533900752ba4cac233d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:01.233138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:01.296232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459daebc69b64533900752ba4cac233d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:01.297319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459daebc69b64533900752ba4cac233d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:01.298362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459daebc69b64533900752ba4cac233d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:01.312399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459daebc69b64533900752ba4cac233d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:01.324007 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459daebc69b64533900752ba4cac233d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:01.345214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1200/2391] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:33:01 2022] DELETE /identity/v3/users/459daebc69b64533900752ba4cac233d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:01.351184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:01.351777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:01.352121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:01.378249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:01.379481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:01.458006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1192/2392] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:01.469924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:01.472809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:01.532637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:01.560702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00495dcb-151c-4234-b712-fb46a56f7692 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:01.567309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:01.568480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:01.568971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:01.569453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:01.585047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:01.586851 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:01.598573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1201/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:01.944475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:01.946355 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:01.961357 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62b2c-ea46-4d12-bf2a-3590d82efdcd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:01.966749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:01.967560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:01.967846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:01.968130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] PATH_INFO: `/v3/projects/105d46d520cb4a2191dda7258a2c8916` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:01.969620 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=105d46d520cb4a2191dda7258a2c8916)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:01.971711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:02.029681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105d46d520cb4a2191dda7258a2c8916', 'request_id': 'req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.030962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:02.039905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105d46d520cb4a2191dda7258a2c8916', 'request_id': 'req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.049972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105d46d520cb4a2191dda7258a2c8916', 'request_id': 'req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.063436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] The token cache is being invalidate because project 105d46d520cb4a2191dda7258a2c8916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:33:02.063869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.064782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.066817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1193/2394] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:33:01 2022] DELETE /identity/v3/projects/105d46d520cb4a2191dda7258a2c8916 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:02.071381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:02.085264 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c429-cb40-40f3-ad66-2f7058a348c4 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:02.167359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:02.168295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:02.168565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:02.168853 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] PATH_INFO: `/v3/users/d54136a637ad475fb52bdac5a9c981c9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:02.170187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54136a637ad475fb52bdac5a9c981c9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:02.172182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:02.229058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54136a637ad475fb52bdac5a9c981c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.230027 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54136a637ad475fb52bdac5a9c981c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.230973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54136a637ad475fb52bdac5a9c981c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.241475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54136a637ad475fb52bdac5a9c981c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.251686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e745faaf-6a75-4f3a-85b2-f25f221a94a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54136a637ad475fb52bdac5a9c981c9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.277012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1202/2395] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:33:02 2022] DELETE /identity/v3/users/d54136a637ad475fb52bdac5a9c981c9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:02.660259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:02.661882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:02.677991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a1b3a-fd8b-483d-b934-2d3cf70c12f2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:02.829086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:02.832072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:02.833320 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:02.834069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] PATH_INFO: `/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:02.836566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5064a8ee8a04a5196fcadabe4a20bc6)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:02.840726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:02.943280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5064a8ee8a04a5196fcadabe4a20bc6', 'request_id': 'req-2483224c-08e2-46c3-9dae-6c2f8a61c01b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.950362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5064a8ee8a04a5196fcadabe4a20bc6', 'request_id': 'req-2483224c-08e2-46c3-9dae-6c2f8a61c01b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.960684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5064a8ee8a04a5196fcadabe4a20bc6', 'request_id': 'req-2483224c-08e2-46c3-9dae-6c2f8a61c01b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.973533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] The token cache is being invalidate because project b5064a8ee8a04a5196fcadabe4a20bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:33:02.973871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.974591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2483224c-08e2-46c3-9dae-6c2f8a61c01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:02.976452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1194/2396] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:33:02 2022] DELETE /identity/v3/projects/b5064a8ee8a04a5196fcadabe4a20bc6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:33:02.988704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:02.988997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:02.989280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.008467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:03.009764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.069895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:03.089393 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1203/2397] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:03.091465 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.091757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.092070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.113782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:03.115206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.128381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1195/2398] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:33:03.129956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.144829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb675-5646-4c79-be6f-88c0cfd94305 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.152110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.153407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.153993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.154559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.156025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:03.156631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.159667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.181643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1204/2399] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:03.186260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.201807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f237-d734-4008-ac04-847e50b3de58 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.210957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.212042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.212665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.212934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.214162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.217404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.254762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1196/2400] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:03 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:03.258700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.270964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f045b-6479-43d6-991a-fde2dc0c44b6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.345804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.346516 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.346805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.347086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.347821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.349909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.374919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1205/2401] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:03 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:03.379319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.393569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6390c-0a05-4112-ba76-2a39d716e7e3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.398941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.399641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.399962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.400247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.401047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.402757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.421993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1197/2402] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:03.426395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.437738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c2192-b2e9-43e9-894e-a2ee6039ed74 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.442926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.443634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.443923 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.444407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.445217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.447300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5091057-5e13-456e-b060-42dac4146023 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.463835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1206/2403] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:03.468297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.488732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3574eb-63f0-49c2-bb11-a9a7302b29d5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.495875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.497400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.497907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.498399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] PATH_INFO: `/v3/projects/9954040eb98d4ecdafbc681132689ff5/users/b35218bf5d284771b5107601ec633f7c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.527693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9954040eb98d4ecdafbc681132689ff5, user_id=b35218bf5d284771b5107601ec633f7c, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.535655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f9588-34d8-430e-9876-88d9ef3ae70c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.555943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1198/2404] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:03 2022] PUT /identity/v3/projects/9954040eb98d4ecdafbc681132689ff5/users/b35218bf5d284771b5107601ec633f7c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:03.563275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.563643 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.563878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.588394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:03.598816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.629234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e25e-8366-4057-bd0d-df4f356a2a9b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.629766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e25e-8366-4057-bd0d-df4f356a2a9b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.630178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2e25e-8366-4057-bd0d-df4f356a2a9b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.632900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1199/2405] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:33:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:03.645195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.645741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.646247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.656694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:03.671718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:03.674682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.719755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1207/2406] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:03.734455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.738598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.788816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1200/2407] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:33:03.802478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:03.803930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:03.835015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba6de1-c83b-4478-ac1b-3faf981dbf97 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.841792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.845615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.846787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.853899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.872783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.875888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6914257-a94f-4e4c-a166-9ba62bf809ca service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.902090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1208/2408] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:03.949680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae9e7f-7350-4672-8a39-0fee6b233482 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:03.954086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:03.955141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:03.955877 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:03.956396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:03.971982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:03.973573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc497-bc42-4853-bc6b-a0b9de94b62f service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:03.994509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1201/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:33:04.078261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.078892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.079256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.080899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1209/2410] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:33:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:04.087836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b26b5-d8c8-4b07-b29c-7d594c74a505 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.087836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b26b5-d8c8-4b07-b29c-7d594c74a505 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.087836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b26b5-d8c8-4b07-b29c-7d594c74a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.110025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-226b26b5-d8c8-4b07-b29c-7d594c74a505 None None] MFA Rules not processed for user `16969e4969c4457eaec3ed37c5757aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:04.111459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-226b26b5-d8c8-4b07-b29c-7d594c74a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:04.118771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1202/2411] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:33:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:33:04.139582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:04.143603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:04.170033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11f0e4-eb33-4115-b9b2-688b8dd83152 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:04.178163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:04.179132 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.179639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.180122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.189640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:04.193976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:04.196385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa922fb-4e6f-4185-8f9d-c6b66f7f5148 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:04.207789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1210/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:04.299464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a69f3-4049-4332-9ad0-e0b05f96049e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.299797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a69f3-4049-4332-9ad0-e0b05f96049e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.300023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a69f3-4049-4332-9ad0-e0b05f96049e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.301589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1203/2413] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jun 27 15:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:04.324518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474f93-40fc-405e-8d40-f282470ba160 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.324940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474f93-40fc-405e-8d40-f282470ba160 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.325374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474f93-40fc-405e-8d40-f282470ba160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.348948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3474f93-40fc-405e-8d40-f282470ba160 None None] MFA Rules not processed for user `c3f4d2d7923a4a9a8f59968f770949d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:04.350619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3474f93-40fc-405e-8d40-f282470ba160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:04.362673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1211/2414] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jun 27 15:33:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:33:04.374200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a69f3-4049-4332-9ad0-e0b05f96049e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:04.387051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a69f3-4049-4332-9ad0-e0b05f96049e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:04.392629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:04.394003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:04.394402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:04.394703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:04.409307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:04.410813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:04.423408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1204/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:07.415338 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a9710-276c-4537-b1a0-2a1658681870 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:07.416669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a9710-276c-4537-b1a0-2a1658681870 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:07.417142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a9710-276c-4537-b1a0-2a1658681870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:07.440240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-208a9710-276c-4537-b1a0-2a1658681870 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:07.441979 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-208a9710-276c-4537-b1a0-2a1658681870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:07.540990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1212/2416] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:33:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:07.558294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:07.561438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:07.694798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:07.704619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8328ea0-36d7-4004-aa7f-eb62f3c5a28c service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:07.714573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:07.715810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:07.716493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:07.717210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:07.734728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:07.739092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:07.755139 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1205/2417] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:33:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:33:11.339085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:11.339613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:11.339708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:11.398729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] MFA Rules not processed for user `b35218bf5d284771b5107601ec633f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:11.400073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:11.622764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1213/2418] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:11.623563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:11.638695 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db817d-5ab8-4782-8778-7b31a90cd473 service swift] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:11.652966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:11.654839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:11.655138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:11.655420 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:11.658633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:11.662634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:11.788993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1206/2419] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:11 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:33:11.794206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:11.839317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069da7d3-f060-451f-bfe9-ad31dc6608d2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:12.100178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:12.101026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:12.101351 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:12.101653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:12.102661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:12.105187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:12.144616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1214/2420] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:11 2022] POST /identity/v3/users => generated 292 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:12.150686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:12.167423 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540afa-c0de-40e3-8135-e19fde94f48e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:12.198624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:12.199550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:12.199875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:12.200096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:12.200933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:12.202655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:12.225231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1207/2421] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:12.232071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:12.261754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa925bb3-32b1-456d-93fd-358a01f022c2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:12.269593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:12.270982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:12.271720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:12.272168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] PATH_INFO: `/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4/users/ca5afb0b572d4ca3b6f7094ada8bc622/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:12.292791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:12.307928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d3dc0907f54b7d9896aa1eef5d0df4, user_id=ca5afb0b572d4ca3b6f7094ada8bc622, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:12.312668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:12.339944 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1215/2422] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:12 2022] PUT /identity/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4/users/ca5afb0b572d4ca3b6f7094ada8bc622/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:12.345682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:12.362913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a5982-d62d-4a85-9b53-0992e6a95573 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:12.493444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:12.494678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:12.494678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:12.494678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:12.495388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:12.497318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:12.513340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1208/2423] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:12.515259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:12.533336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4a71b-7aee-428b-a41e-73abed48ffd4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:12.539341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:12.540567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:12.540926 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:12.541270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] PATH_INFO: `/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4/users/ca5afb0b572d4ca3b6f7094ada8bc622/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:12.544047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d3dc0907f54b7d9896aa1eef5d0df4, user_id=ca5afb0b572d4ca3b6f7094ada8bc622, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:12.547602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:12.568370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1216/2424] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:12 2022] PUT /identity/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4/users/ca5afb0b572d4ca3b6f7094ada8bc622/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:15.099240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:15.102513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:15.165718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:15.167689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:15.364005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97778958-4d36-46cd-addf-407a28f9aced admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:15.370052 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:15.370725 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:15.371007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:15.371281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:15.373996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:15.378970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d1bc4-eb51-4454-b90c-3e92015f26b7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:15.390706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:15.391587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:15.392017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:15.393410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:15.398750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:15.576659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:15.578333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4b6ab-565d-4800-9467-e23f6f47b20c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:15.587377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:15.591675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:15.604510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1217/2425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:15.633326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1209/2426] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:21.050324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac36b-8dcf-4a67-a580-d2198dc3eba5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:21.050911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac36b-8dcf-4a67-a580-d2198dc3eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:21.051495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac36b-8dcf-4a67-a580-d2198dc3eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:21.090794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5f5ac36b-8dcf-4a67-a580-d2198dc3eba5 None None] MFA Rules not processed for user `ca5afb0b572d4ca3b6f7094ada8bc622`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:21.094457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f5ac36b-8dcf-4a67-a580-d2198dc3eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:21.234314 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1218/2427] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:21.249210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:21.251028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:21.262755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:21.374136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430896e0-71f0-4d08-9dd1-027fe607225e service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:21.380125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:21.380841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:21.381122 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:21.381390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:21.473020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:21.474696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430a09e-13db-4a97-930d-e30ef3f7607c service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:21.485658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1210/2428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:23.165962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:23.166854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:23.166854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:23.189542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] MFA Rules not processed for user `b35218bf5d284771b5107601ec633f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:23.191468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:23.200207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1219/2429] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:27.586435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9cae-ba45-4437-8db0-48af0f186244 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:27.586964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9cae-ba45-4437-8db0-48af0f186244 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:27.589299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9cae-ba45-4437-8db0-48af0f186244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:27.617040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5d9c9cae-ba45-4437-8db0-48af0f186244 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:27.618611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5d9c9cae-ba45-4437-8db0-48af0f186244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:27.720958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1211/2430] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:27.749598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:27.763727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:27.891134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2fffd-ca7e-4f8b-95f8-c792d6a04f95 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:27.901808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:27.902816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:27.903422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:27.904558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:27.915389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:27.917473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:27.934841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1220/2431] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:33:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:33:31.393586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.394439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.394439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.414727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] MFA Rules not processed for user `23abc809ab644c1da0270a35ac8bab21`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:31.415985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:31.424818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1212/2432] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:33.210378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:33.277890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbb4b-bf73-4a8e-bf2b-7c52427a4d56 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:33.281779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:33.282433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:33.282829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:33.283789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:33.285756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:33.360973 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:33.363556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:33.375935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1221/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:38.557187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:38.560299 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:38.660928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:38.663348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:38.664278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:38.685577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7317a5-99f0-480a-b18e-3acd9023ed50 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:38.690846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:38.691857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:38.692418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:38.692858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] PATH_INFO: `/v3/users/995fa1760f5241588a493e5f894b0e3f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:38.694175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=995fa1760f5241588a493e5f894b0e3f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:38.696458 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:38.697762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4908a50-f288-4561-b48d-6c6dee1d2fbb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:38.705145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:38.706319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:38.706801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:38.707282 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:38.710382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:38.730729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:38.734748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:38.752970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1213/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:38.759980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:38.765321 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995fa1760f5241588a493e5f894b0e3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:38.768910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995fa1760f5241588a493e5f894b0e3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:38.769858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995fa1760f5241588a493e5f894b0e3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:38.778959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995fa1760f5241588a493e5f894b0e3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:38.791066 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ac1f71c7-a40d-4109-805d-ce688ac61240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995fa1760f5241588a493e5f894b0e3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:38.815357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1222/2435] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:33:38 2022] DELETE /identity/v3/users/995fa1760f5241588a493e5f894b0e3f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:39.022545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.036371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965f8df-5aa7-4e0d-921f-0c504e8799b0 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.110079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.111667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.111946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.112305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] PATH_INFO: `/v3/projects/2e07fc46cd5f4afe875767c926bd63ee` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.113574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e07fc46cd5f4afe875767c926bd63ee)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.115498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.194286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e07fc46cd5f4afe875767c926bd63ee', 'request_id': 'req-c5e6e96b-736d-4869-8b85-b4f4a5f21288'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:39.204391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e07fc46cd5f4afe875767c926bd63ee', 'request_id': 'req-c5e6e96b-736d-4869-8b85-b4f4a5f21288'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:39.215992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e07fc46cd5f4afe875767c926bd63ee', 'request_id': 'req-c5e6e96b-736d-4869-8b85-b4f4a5f21288'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:39.229413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] The token cache is being invalidate because project 2e07fc46cd5f4afe875767c926bd63ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:33:39.229822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:39.230570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c5e6e96b-736d-4869-8b85-b4f4a5f21288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:39.232278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1214/2436] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:33:39 2022] DELETE /identity/v3/projects/2e07fc46cd5f4afe875767c926bd63ee => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:39.237702 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.238077 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.239022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.261513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:39.267439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:39.345009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1223/2437] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:33:39.355065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.355065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.355065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.373962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:39.375340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:39.388267 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1215/2438] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:39.389331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.400106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204448af-ab75-49d2-93d6-08099572cd97 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.405446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.406147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.406467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.406738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.407485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:39.408201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.410160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.424859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1224/2439] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:39.428387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.442043 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909abe-9576-4586-80c4-0f15c6ba8d6f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.448136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.448850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.449135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.449413 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.450206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.452228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.483127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1216/2440] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:39.489427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.500755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813d42-eb77-4bfc-aae7-61ec5531330f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.578523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.579755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.580100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.580550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.581716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.583721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.606615 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1225/2441] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:39.617639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.629615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6969d-5d02-431c-89b4-70e7f2618ac8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.634446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.635149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.635428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.635712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.636496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.638189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.649262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1217/2442] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:39.653721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.665375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5991c3-950d-462c-9171-470b03ade3bb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.670431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.671107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.671389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.671659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.672444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.674142 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd95f59-77db-4b60-acab-f4cdf36bbf99 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.686767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1226/2443] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:39.691870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.705989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8920db-a0a6-4923-9450-f3ea75f0aeed admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.713934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.715229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.715516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.715794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] PATH_INFO: `/v3/projects/b7d8c30b339f42309d1b00ad32f3694d/users/4dc4d0d4900a4b8e8720218b8402ce19/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.734390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d8c30b339f42309d1b00ad32f3694d, user_id=4dc4d0d4900a4b8e8720218b8402ce19, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.737404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.746711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.756886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1218/2444] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:39 2022] PUT /identity/v3/projects/b7d8c30b339f42309d1b00ad32f3694d/users/4dc4d0d4900a4b8e8720218b8402ce19/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:39.774477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.774797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.775113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.811081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] MFA Rules not processed for user `4dc4d0d4900a4b8e8720218b8402ce19`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:39.812297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:39.828728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.913143 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1227/2445] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:39.914779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.036705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215265a9-7462-4eee-81cc-4fc968c9cfc9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.036705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.036705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.036705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.037254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.038040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.040284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.076374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1219/2446] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:40.078257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.094616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22bb06-5328-4aa0-bbef-dec866ba9c7d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.171509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.172431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.172705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.173054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.174634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.176935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.200352 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1228/2447] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:40 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:40.203849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.216674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e645d5-bdf3-445b-a4a9-99313f34bc51 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.223131 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.223960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.224248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.224628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.225285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.227164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.240370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1220/2448] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:40.244965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.256661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c71bb-a25a-4510-86f8-7290eca26774 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.262144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.263145 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.263428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.263857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] PATH_INFO: `/v3/projects/fd02b1cf49dc435988bb9a98709e25b1/users/84e4511f19e3456a813ea5784e9461d8/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.281422 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd02b1cf49dc435988bb9a98709e25b1, user_id=84e4511f19e3456a813ea5784e9461d8, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.284678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.309276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1229/2449] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:40 2022] PUT /identity/v3/projects/fd02b1cf49dc435988bb9a98709e25b1/users/84e4511f19e3456a813ea5784e9461d8/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:40.323045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.336120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe60ee-455b-4b0d-ae16-d3900739b4f1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.410020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.410801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.411086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.411364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.412138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.414022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1cd21-a2bd-4662-ac1c-57c4f9d7e4ec admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.425627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1221/2450] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:40.430085 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.441800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b334e4f-2f94-4b0f-8f8f-399eb0df81fa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.447082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.448035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.448334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.449498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] PATH_INFO: `/v3/projects/fd02b1cf49dc435988bb9a98709e25b1/users/84e4511f19e3456a813ea5784e9461d8/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.451972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd02b1cf49dc435988bb9a98709e25b1, user_id=84e4511f19e3456a813ea5784e9461d8, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.454880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.467014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1230/2451] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:40 2022] PUT /identity/v3/projects/fd02b1cf49dc435988bb9a98709e25b1/users/84e4511f19e3456a813ea5784e9461d8/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:40.479180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.479477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.479757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.508569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] MFA Rules not processed for user `84e4511f19e3456a813ea5784e9461d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:40.509815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:40.573633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:40.587632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1222/2452] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:40.600020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.602122 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:40.666174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6ff4f-e5bc-479a-854e-2015d23d221d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.670217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.670876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.671156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.671429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.682985 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.684725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.694826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1231/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:33:40.780611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:40.792469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c6788a-f5ff-40fa-84d5-de43871904a7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:40.796954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:40.797730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:40.798018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:40.798345 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:40.861964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:40.863805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70beffd6-be40-4e1c-bd52-3f1c03b736f2 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:40.877550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1223/2454] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:50.594499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.613391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:50.674369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12702147-51a4-4515-8c40-b2bd9c80a435 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.678984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.679891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.680335 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.680637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] PATH_INFO: `/v3/users/4dc4d0d4900a4b8e8720218b8402ce19` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.681847 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc4d0d4900a4b8e8720218b8402ce19)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.684174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.740074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc4d0d4900a4b8e8720218b8402ce19'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:50.740775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc4d0d4900a4b8e8720218b8402ce19'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:50.741375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc4d0d4900a4b8e8720218b8402ce19'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:50.749568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc4d0d4900a4b8e8720218b8402ce19'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:50.759524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc4d0d4900a4b8e8720218b8402ce19'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:50.784778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1232/2455] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:33:50 2022] DELETE /identity/v3/users/4dc4d0d4900a4b8e8720218b8402ce19 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:50.789690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.790183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.790305 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.810661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:50.811963 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:50.895447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1224/2456] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:50.905370 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.906830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:50.968200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2adfe-822b-4155-996e-7e922ffa6ff2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.972101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.972762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.973072 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.973309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.983504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.984940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.995796 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1233/2457] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:51.401290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.402989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:51.425270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4faf7d-09b9-498d-8235-66ddd6883960 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.431183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.432169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.433301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.433572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] PATH_INFO: `/v3/projects/b7d8c30b339f42309d1b00ad32f3694d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.435222 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d8c30b339f42309d1b00ad32f3694d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:51.437950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:51.492750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d8c30b339f42309d1b00ad32f3694d', 'request_id': 'req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.499603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d8c30b339f42309d1b00ad32f3694d', 'request_id': 'req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.509251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d8c30b339f42309d1b00ad32f3694d', 'request_id': 'req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.521462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] The token cache is being invalidate because project b7d8c30b339f42309d1b00ad32f3694d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:33:51.521745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.522375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.523917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1225/2458] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:33:51 2022] DELETE /identity/v3/projects/b7d8c30b339f42309d1b00ad32f3694d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:51.528025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.538311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d40c5-9971-40d0-a20d-40afa89cd49e service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.569543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.597834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.598710 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.598983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.599258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] PATH_INFO: `/v3/users/84e4511f19e3456a813ea5784e9461d8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.600503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e4511f19e3456a813ea5784e9461d8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:51.602443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:51.652411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e4511f19e3456a813ea5784e9461d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.653078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e4511f19e3456a813ea5784e9461d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.653706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e4511f19e3456a813ea5784e9461d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.660908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e4511f19e3456a813ea5784e9461d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.670634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6de06de6-15f4-4e5c-80e1-61cf7447a30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e4511f19e3456a813ea5784e9461d8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:51.692648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1234/2459] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:33:51 2022] DELETE /identity/v3/users/84e4511f19e3456a813ea5784e9461d8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:52.007760 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.010045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.025069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc2d7-c02c-40c0-92f1-8f13ee39f848 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.039893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.099041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.100126 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.100430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.100738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] PATH_INFO: `/v3/projects/fd02b1cf49dc435988bb9a98709e25b1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.102105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd02b1cf49dc435988bb9a98709e25b1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.104141 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.164440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd02b1cf49dc435988bb9a98709e25b1', 'request_id': 'req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:52.171560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd02b1cf49dc435988bb9a98709e25b1', 'request_id': 'req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:52.183417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd02b1cf49dc435988bb9a98709e25b1', 'request_id': 'req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:52.194972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] The token cache is being invalidate because project fd02b1cf49dc435988bb9a98709e25b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:33:52.195261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:52.195962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdd99832-a25b-4c4d-9d75-fc36d0a8ed6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:33:52.197563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1226/2460] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:33:52 2022] DELETE /identity/v3/projects/fd02b1cf49dc435988bb9a98709e25b1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:52.202662 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.202955 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.203235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.223134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:52.224344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.299953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1235/2461] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:52.309279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.309571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.309843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.331754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:52.333119 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.342794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1227/2462] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:52.348966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.366774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff9be-2843-4667-8b5d-584180b10ff4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.374333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.375484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.375990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.376509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.377899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:52.378298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.381367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.406208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1236/2463] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.407541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.418861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad9c60-55bb-4c90-b6b6-1cc52dbf63a1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.423855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.424901 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.425270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.425560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.426279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.428248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.466632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1228/2464] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:52.467891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.481436 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f1316-e785-4eb2-8e69-9422dd097c0f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.524816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.546990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.547688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.547976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.548254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.549287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.551309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.574819 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1237/2465] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:33:52.579183 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.590972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2ac2-1f69-4d80-a37a-88cfc8f3e571 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.596545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.597223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.597518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.597783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.598515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.600128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.611908 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1229/2466] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:52.615957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.626697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45fbf7-2c82-46a6-b18b-546d8c1a7dd0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.631890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.632592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.632884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.633162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.633931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.635568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbf749-3c21-44a9-bde6-32d05ac6f8cb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.650735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1238/2467] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:52.655778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.667405 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff5132-4307-41d5-8330-5e7361f9ecbf admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.672909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.673829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.674089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.675018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] PATH_INFO: `/v3/projects/b8e4337808b843659e0efbc886527150/users/cb824e7313444e09b79e93ae84c6a7cb/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.692488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e4337808b843659e0efbc886527150, user_id=cb824e7313444e09b79e93ae84c6a7cb, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.695221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.710915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1230/2468] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:52 2022] PUT /identity/v3/projects/b8e4337808b843659e0efbc886527150/users/cb824e7313444e09b79e93ae84c6a7cb/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:52.722125 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.722383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.722693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.753235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] MFA Rules not processed for user `cb824e7313444e09b79e93ae84c6a7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:52.754523 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.826366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1239/2469] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:52.831866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.881974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.905090 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26760c-0ba0-404e-ad26-515b21adef2e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.910801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.911476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.911782 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.912056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.912822 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.914763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.944935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1231/2470] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:52.949368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.962492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdd4b2-9576-48d6-969b-433f31481c5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.043479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.044655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.045137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.045624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.046854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.051271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.081903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1240/2471] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:53.086316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.098300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bfc0b-aa75-48d1-9b78-845ecc74724e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.103913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.104766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.105063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.105359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.106248 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.108249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.121046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1232/2472] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:53.125478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.137462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cf2e0-3c34-4985-8c5c-9c82d077a833 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.142313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.143939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.144251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.144561 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] PATH_INFO: `/v3/projects/609d93e91c804ae7a0710b7c4b80714d/users/d8d9b2bac5884cb1ad02b808a5cd949b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.161732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609d93e91c804ae7a0710b7c4b80714d, user_id=d8d9b2bac5884cb1ad02b808a5cd949b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.164673 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.180816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1241/2473] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/609d93e91c804ae7a0710b7c4b80714d/users/d8d9b2bac5884cb1ad02b808a5cd949b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:33:53.185441 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.199181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68184590-0181-4596-b0ae-90642b420d61 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.275457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.276300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.276826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.277336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.278755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.280602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748a912-9b17-44a7-86bd-12d61424c892 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.296445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1233/2474] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:33:53.301501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.313190 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83421604-a5b6-4bd6-9533-78085420ccd7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.320070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.321087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.321368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.321641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] PATH_INFO: `/v3/projects/609d93e91c804ae7a0710b7c4b80714d/users/d8d9b2bac5884cb1ad02b808a5cd949b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.323813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=609d93e91c804ae7a0710b7c4b80714d, user_id=d8d9b2bac5884cb1ad02b808a5cd949b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.326513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.334667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.341449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1242/2475] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/609d93e91c804ae7a0710b7c4b80714d/users/d8d9b2bac5884cb1ad02b808a5cd949b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:53.352535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.352826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.353105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.384027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] MFA Rules not processed for user `d8d9b2bac5884cb1ad02b808a5cd949b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:53.385307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:53.468924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1234/2476] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:33:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:33:53.480869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.482599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:53.563575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecd0-bde4-4246-96f5-eac70cfe4cfe admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.568018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.569196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.569480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.569759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.640030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.641838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.654159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1243/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:33:58.379818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.390265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789e8f3-904b-4778-99dd-e54f2950b77b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.394984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.395239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.395527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.395804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.406539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:58.408193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:58.424561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1235/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:03.391317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:03.400974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:03.467502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38248e57-c5e6-4955-b27a-4c041323e274 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:03.472653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:03.473315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:03.473603 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:03.473879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:03.475795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:03.544021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:03.545166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:03.556718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1244/2479] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:08.414805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:13.551249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:19.810071 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:19.820891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a855-9255-4811-a0b9-1cc5e770c8da service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:19.825085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:19.825745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.826029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.826306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.828266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:19.896949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:19.898396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54a3ee-9b1f-4dde-b2df-a93f3c370232 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:19.908889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1236/2480] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:22.749782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:22.763679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e19baa-a1be-477b-b107-127e7467cef6 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:22.768935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:22.769860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.770147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.770419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] PATH_INFO: `/v3/users/cb824e7313444e09b79e93ae84c6a7cb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:22.772283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb824e7313444e09b79e93ae84c6a7cb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:22.774525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:22.842231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb824e7313444e09b79e93ae84c6a7cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:22.844501 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb824e7313444e09b79e93ae84c6a7cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:22.851078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb824e7313444e09b79e93ae84c6a7cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:22.862457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb824e7313444e09b79e93ae84c6a7cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:22.879134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb824e7313444e09b79e93ae84c6a7cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:22.909430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1245/2481] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:22 2022] DELETE /identity/v3/users/cb824e7313444e09b79e93ae84c6a7cb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:22.912582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.913111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.913683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:22.941659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:22.944241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.020150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1237/2482] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:23.027208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.028929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.091057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baad88-0f2d-4842-a9aa-f502a0c6cf73 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.095019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.095708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.095989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.096259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.106295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.107765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.118153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1246/2483] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:23.457020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.458650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.468530 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.470095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.478722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be602117-1cf5-4b5d-bb11-8b86403a113a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.483805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.484761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.485092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.485209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a0b1-c313-4484-8d7f-71b306218c49 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.485319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] PATH_INFO: `/v3/users/e9dab30db0a44655828384db53df29f9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.486775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9dab30db0a44655828384db53df29f9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.491058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.492467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.492627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.492831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.493108 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] PATH_INFO: `/v3/projects/b8e4337808b843659e0efbc886527150` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.494366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e4337808b843659e0efbc886527150)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.496560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.555723 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e4337808b843659e0efbc886527150', 'request_id': 'req-0d80f0bc-89cc-45bd-bde1-a0872260f479'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.563205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e4337808b843659e0efbc886527150', 'request_id': 'req-0d80f0bc-89cc-45bd-bde1-a0872260f479'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.564096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9dab30db0a44655828384db53df29f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.571722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e4337808b843659e0efbc886527150', 'request_id': 'req-0d80f0bc-89cc-45bd-bde1-a0872260f479'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.572205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9dab30db0a44655828384db53df29f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.582416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.589916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] The token cache is being invalidate because project b8e4337808b843659e0efbc886527150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:23.590263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.590876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9dab30db0a44655828384db53df29f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.591438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.592108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9dab30db0a44655828384db53df29f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.593065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9dab30db0a44655828384db53df29f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.593928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1247/2484] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:23 2022] DELETE /identity/v3/projects/b8e4337808b843659e0efbc886527150 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:23.598799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.601360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.619560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1238/2485] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:23 2022] DELETE /identity/v3/users/e9dab30db0a44655828384db53df29f9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:23.675299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80f0bc-89cc-45bd-bde1-a0872260f479 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.735117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.736006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.736288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.736593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] PATH_INFO: `/v3/users/d8d9b2bac5884cb1ad02b808a5cd949b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.737801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d9b2bac5884cb1ad02b808a5cd949b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.739983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.788389 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d9b2bac5884cb1ad02b808a5cd949b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.789070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d9b2bac5884cb1ad02b808a5cd949b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.789621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d9b2bac5884cb1ad02b808a5cd949b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.797074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.797527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d9b2bac5884cb1ad02b808a5cd949b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.799625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.806677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed36f5f-2107-43ae-af08-1d6aa238b1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d9b2bac5884cb1ad02b808a5cd949b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.823232 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1fbf-0c9f-4d42-8cd9-0dbddc0dabbc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.827470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1248/2486] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:23 2022] DELETE /identity/v3/users/d8d9b2bac5884cb1ad02b808a5cd949b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:23.836364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.837607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.837893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.838172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] PATH_INFO: `/v3/projects/2270daf238a640609e9ab514b01d9768` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.839487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2270daf238a640609e9ab514b01d9768)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.841678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.843459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.905946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2270daf238a640609e9ab514b01d9768', 'request_id': 'req-4f759bd3-97f8-4766-8dbc-1f1eddece229'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.916582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2270daf238a640609e9ab514b01d9768', 'request_id': 'req-4f759bd3-97f8-4766-8dbc-1f1eddece229'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.924622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2270daf238a640609e9ab514b01d9768', 'request_id': 'req-4f759bd3-97f8-4766-8dbc-1f1eddece229'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.936570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] The token cache is being invalidate because project 2270daf238a640609e9ab514b01d9768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:23.936570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.937633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4f759bd3-97f8-4766-8dbc-1f1eddece229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:23.939912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1239/2487] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:23 2022] DELETE /identity/v3/projects/2270daf238a640609e9ab514b01d9768 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:23.945440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.945646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.945861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.967160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:23.968492 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.001959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.061632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1249/2488] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:24.070702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.070988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.071260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.090529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.091753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.101197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1240/2489] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:24.105265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.115873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff191a1-fdd8-401b-8996-c0fef2f1f982 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.122419 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.123207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.123509 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.123790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.124604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:24.124882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.126716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.141552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.142954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1250/2490] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:24.143096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.148048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.156209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad7a58-d42f-463a-b00c-57f17aa64a1b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.159385 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2b737-9883-466a-9a42-ef1142487ebd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.162524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.163649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.163932 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.164171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] PATH_INFO: `/v3/projects/609d93e91c804ae7a0710b7c4b80714d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.164854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.165594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.165849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609d93e91c804ae7a0710b7c4b80714d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.165983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.166333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.167097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.168214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.169216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.201465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1251/2491] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:24.205956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.226221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609d93e91c804ae7a0710b7c4b80714d', 'request_id': 'req-9c799a68-2496-4ac3-bb1a-aeeebe232e69'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:24.232934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609d93e91c804ae7a0710b7c4b80714d', 'request_id': 'req-9c799a68-2496-4ac3-bb1a-aeeebe232e69'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:24.242371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609d93e91c804ae7a0710b7c4b80714d', 'request_id': 'req-9c799a68-2496-4ac3-bb1a-aeeebe232e69'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:24.257467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] The token cache is being invalidate because project 609d93e91c804ae7a0710b7c4b80714d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:24.257759 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:24.258417 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9c799a68-2496-4ac3-bb1a-aeeebe232e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:24.260085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1241/2492] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:24 2022] DELETE /identity/v3/projects/609d93e91c804ae7a0710b7c4b80714d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:24.266539 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaf3a-85e4-4652-8213-f4e2e7ddf598 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.266830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaf3a-85e4-4652-8213-f4e2e7ddf598 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.267113 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaf3a-85e4-4652-8213-f4e2e7ddf598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.284222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8018-b7d8-498a-a063-25cdf54d9fc6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.292399 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e78eaf3a-85e4-4652-8213-f4e2e7ddf598 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.293910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e78eaf3a-85e4-4652-8213-f4e2e7ddf598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.353203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.353766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.354047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.354319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.355020 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.357014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.360557 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1242/2493] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:24.368387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.368670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.368950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.379950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1252/2494] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:24.382255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.384293 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.386400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.389809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.391189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.399808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80164ca2-cad3-4dfe-90ac-793e3c22f8ea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.401772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1243/2495] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:24.404801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.405482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.405759 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.406033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.406158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.406801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.408426 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.415673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a127d92-3038-4c46-83ff-428167dff434 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.419657 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1253/2496] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:24.420431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.421093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.421376 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.421654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.422382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:24.422689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.423932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.424352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.434788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c83c80-3919-44e6-9f1e-24560370b2a4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.436986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1244/2497] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:24.439589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.440248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.440559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.440860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.441442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.441591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.444141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.452817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c110ed-d220-4169-81c3-77708a1578f7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.456202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1254/2498] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:24.458271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.459771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.460554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.460783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.460903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.461623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.463797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.466908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.471044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c1f25-49e6-4a11-9c12-7bde1a25c833 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.477148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.478039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.478320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.478724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] PATH_INFO: `/v3/projects/762136334e4c4c1491cf12d328716123/users/d2154473d9fe4225ab0cdcaa7e67ea0c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.492633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1245/2499] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:24.494773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762136334e4c4c1491cf12d328716123, user_id=d2154473d9fe4225ab0cdcaa7e67ea0c, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.496940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.497592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b92acc-7af4-4906-a73a-a8933cbdfb70 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.511347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1255/2500] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:24 2022] PUT /identity/v3/projects/762136334e4c4c1491cf12d328716123/users/d2154473d9fe4225ab0cdcaa7e67ea0c/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:24.522038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.522331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.522609 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.548870 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] MFA Rules not processed for user `d2154473d9fe4225ab0cdcaa7e67ea0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.550160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.568767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353a145-4d08-4a4d-9b48-6289eb618efe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.618030 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1256/2501] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:24.629449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.630964 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.632938 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.633625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.633909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.634180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.634865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.637046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.658433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1246/2502] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:24.663314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.673779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451396a-d298-466d-b75c-08369218a020 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.678711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.679412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.679668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.679949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.680708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.682704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.696018 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1247/2503] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:24.698178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.708572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36b63a-f19b-4211-a5bb-b7eec93f9181 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.710617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd73611-2cfe-4bd3-85bd-6846eccc9de8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.714243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.715025 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.715498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.715841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.716654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.717362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.718087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.718490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.718710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.718912 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.729211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.730903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5fcf-cac2-462b-b182-16750fb5747c service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.731060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1248/2504] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:24.735340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.741326 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1257/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:24.745722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0972d-bc91-464f-b70c-85d96490dd20 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.750621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.751507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.751787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.752061 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] PATH_INFO: `/v3/projects/da11f50dd0ce4418b722b5e544ccd0af/users/459425a588ca4f8b97ca7e1f62bc19ef/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.768144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da11f50dd0ce4418b722b5e544ccd0af, user_id=459425a588ca4f8b97ca7e1f62bc19ef, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.771049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.791914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1249/2506] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:24 2022] PUT /identity/v3/projects/da11f50dd0ce4418b722b5e544ccd0af/users/459425a588ca4f8b97ca7e1f62bc19ef/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:24.800626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.801071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.801276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.835814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] MFA Rules not processed for user `459425a588ca4f8b97ca7e1f62bc19ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.836899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.889495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.913533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1258/2507] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:24.918330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.939203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.979572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.994205 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a8cbe-1299-4b4a-9a13-8a88c176ff7f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.999231 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.999916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.000211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.000518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.001224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.003181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.039462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1250/2508] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:25.040630 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.048720 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa216b45-e7e8-47c5-9321-e44c2b8cd20a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.113379 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.114473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.114753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.115040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] PATH_INFO: `/v3/users/d2154473d9fe4225ab0cdcaa7e67ea0c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.116685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2154473d9fe4225ab0cdcaa7e67ea0c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.118680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.133747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4341a52-dcd1-4186-8a26-5db989445b82 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.138618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.139316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.139609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.139888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.140635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.142628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.166419 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1251/2509] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:25.170440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.178727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2154473d9fe4225ab0cdcaa7e67ea0c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.179410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2154473d9fe4225ab0cdcaa7e67ea0c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.180119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2154473d9fe4225ab0cdcaa7e67ea0c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.180378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad972e7f-1b82-4744-bb33-2afd844a935f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.185726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.186518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.186888 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.187260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.188020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.188152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2154473d9fe4225ab0cdcaa7e67ea0c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.189641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.198864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b26466f1-a190-402d-aef1-0809a0de3ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2154473d9fe4225ab0cdcaa7e67ea0c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.201971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1252/2510] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:25.206024 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.208219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.221858 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1259/2511] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:24 2022] DELETE /identity/v3/users/d2154473d9fe4225ab0cdcaa7e67ea0c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:25.226801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.227117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.227403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.228103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cd8a6-55d5-41a0-91df-1427f74231c3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.245306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:25.246573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.252443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.288280 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.289230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.289524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.289801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] PATH_INFO: `/v3/projects/b35e6545272a45628aa021d620c1929a/users/5c4b12e01d42417fbe74f77a6a2d4d3c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.305604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35e6545272a45628aa021d620c1929a, user_id=5c4b12e01d42417fbe74f77a6a2d4d3c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.308865 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.311010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1260/2512] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:25.322783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.325572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.326216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1253/2513] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:25 2022] PUT /identity/v3/projects/b35e6545272a45628aa021d620c1929a/users/5c4b12e01d42417fbe74f77a6a2d4d3c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:25.330050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.334434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.384397 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b1804-b284-4f96-8341-14ee370af209 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.396203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0bc56-a1c2-455c-b753-99060edc4ccd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.401005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.401687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.401975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.402252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.402988 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.404642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.418247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1254/2514] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:25.419295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.431127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24aa0d-d59d-44ad-a1d8-21655e83a483 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.436404 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.437306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.437592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.437868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] PATH_INFO: `/v3/projects/b35e6545272a45628aa021d620c1929a/users/5c4b12e01d42417fbe74f77a6a2d4d3c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.438553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.439217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.439503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.439782 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.439917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35e6545272a45628aa021d620c1929a, user_id=5c4b12e01d42417fbe74f77a6a2d4d3c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.442727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f9d60-81dc-4b6e-ae21-2c5166ba738b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.449484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.450938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.454475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1255/2515] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:25 2022] PUT /identity/v3/projects/b35e6545272a45628aa021d620c1929a/users/5c4b12e01d42417fbe74f77a6a2d4d3c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:25.461583 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1261/2516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:25.466085 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.466373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.466653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.495095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] MFA Rules not processed for user `5c4b12e01d42417fbe74f77a6a2d4d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:25.496298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.578182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1256/2517] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:25.592467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.594433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.662860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8ddd6-ea0f-48db-80ff-730cae8e9e9b service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.688555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.715698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.716376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.716661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.716932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.727837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.729371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.741098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1262/2518] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:25.741287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.742831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.787962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.789425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.801150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9c45-4ec1-4c8c-8fc7-c18af218e9f5 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.805151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.805892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.806236 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.806589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.808799 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.816919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f5ac9-db7a-4ae0-aa30-4e439b853ff5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.821794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.822727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.823012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.823351 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] PATH_INFO: `/v3/projects/762136334e4c4c1491cf12d328716123` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.824558 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762136334e4c4c1491cf12d328716123)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.826520 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.839863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.880479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.881934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9b5b-389c-48c4-ac54-948e35ccefe5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.883362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762136334e4c4c1491cf12d328716123', 'request_id': 'req-f95c08ac-88e8-4b72-95a7-4661f47e71a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.889798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762136334e4c4c1491cf12d328716123', 'request_id': 'req-f95c08ac-88e8-4b72-95a7-4661f47e71a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.890959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1263/2519] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:25.898306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762136334e4c4c1491cf12d328716123', 'request_id': 'req-f95c08ac-88e8-4b72-95a7-4661f47e71a9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.909686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] The token cache is being invalidate because project 762136334e4c4c1491cf12d328716123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:25.910036 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.910729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f95c08ac-88e8-4b72-95a7-4661f47e71a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:25.912511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1257/2520] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:25 2022] DELETE /identity/v3/projects/762136334e4c4c1491cf12d328716123 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:25.917595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.917896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.918141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.937274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:25.938535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:26.010780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1264/2521] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:26.012691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.013201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.013439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.033658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:26.034947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:26.045905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1258/2522] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:26.050133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.060625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfec17f-7d9e-4365-9878-172a0744b934 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.065932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.066804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.067174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.067646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.069119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.069500 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.071206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.084585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1265/2523] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:26.089387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.099242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6834c1-4218-4280-939a-2e728f3772d1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.104528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.105275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.105664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.106092 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.106928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.109418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.142818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1259/2524] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:26.146882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.158962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57f19-5382-4f77-b6b8-89c5708bb97d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.237242 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.245069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.245888 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.246266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.246558 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.247405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.249495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.276731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1266/2525] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:26.279284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.296674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0631be-e867-4af9-ab2c-79792c217a62 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.302408 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.303140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.303527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.303816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.304592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.306364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.322152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1260/2526] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:26.326895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.340655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f432740-5476-469d-9738-ad233b0505b7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.347361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.348173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.348614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.348957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.349941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.351835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f85f6-d500-4e41-82b4-b40724729232 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.362963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1267/2527] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:26.367858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.378969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f0852-6350-4c6f-a69b-0274af08f6ed admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.384833 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.385728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.386008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.386283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] PATH_INFO: `/v3/projects/4bf159f67d8f44209d36afb332f8f6e1/users/e9f856d15afa4077848276eaf432c2aa/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.403048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf159f67d8f44209d36afb332f8f6e1, user_id=e9f856d15afa4077848276eaf432c2aa, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.405774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.423342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1261/2528] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:26 2022] PUT /identity/v3/projects/4bf159f67d8f44209d36afb332f8f6e1/users/e9f856d15afa4077848276eaf432c2aa/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:26.437094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.437388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.437677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.472697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] MFA Rules not processed for user `e9f856d15afa4077848276eaf432c2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:26.474304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:26.538568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1268/2529] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:26.543446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.622359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d5852-7526-485e-9d75-1a0de6c8b7f8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.627251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.627921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.628200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.628512 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.629194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.631391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.669026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1262/2530] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:26.675183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.690989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65558ecc-4428-4c93-9cb4-4c3d75a779b7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.750757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.751448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.751732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.752013 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.752730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.754694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.776968 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1269/2531] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:26 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:26.781255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.793414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd45ced-321f-4fa6-9009-5ba65552dcbc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.799383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.800168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.800583 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.800862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.801595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.803849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.815793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1263/2532] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:26.820933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.831347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7e5c-bb5b-44da-8ba4-1950b0cf1892 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.837053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.837957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.838241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.838515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] PATH_INFO: `/v3/projects/a504bcf1545342b09b1d79cba5e06bb1/users/be189bc20706425fadb5d4dc4dda6e99/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.855613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a504bcf1545342b09b1d79cba5e06bb1, user_id=be189bc20706425fadb5d4dc4dda6e99, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.858312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.877348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1270/2533] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:26 2022] PUT /identity/v3/projects/a504bcf1545342b09b1d79cba5e06bb1/users/be189bc20706425fadb5d4dc4dda6e99/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:26.882521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.893133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdde58-d79e-4376-9487-b0340206a810 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.901959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.959137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.960064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.960548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.960923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.962080 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.964685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945ee62-1b44-44f0-9151-321f88ea5d87 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.978444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1264/2534] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:26.982868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.994524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7717-4a91-41dd-aac1-dc7ee6967351 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.000648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.001568 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.001846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.002133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] PATH_INFO: `/v3/projects/a504bcf1545342b09b1d79cba5e06bb1/users/be189bc20706425fadb5d4dc4dda6e99/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.004514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a504bcf1545342b09b1d79cba5e06bb1, user_id=be189bc20706425fadb5d4dc4dda6e99, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.007295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.019719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1271/2535] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:26 2022] PUT /identity/v3/projects/a504bcf1545342b09b1d79cba5e06bb1/users/be189bc20706425fadb5d4dc4dda6e99/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:27.030578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.031329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.031329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.060605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] MFA Rules not processed for user `be189bc20706425fadb5d4dc4dda6e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:27.061839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:27.133672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1265/2536] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:27.145981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.148166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:27.168986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.212425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043394a1-4860-4110-a050-0df5cbd13176 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.217089 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.217736 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.218021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.218288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.228976 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.230424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a887e8-33f3-4d29-ba42-d6af6a3333f7 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.240788 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1272/2537] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:27.320657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.403997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88492503-ce47-4f59-8e96-d8c05812aee3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.410964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.412130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.412492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.412780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] PATH_INFO: `/v3/users/e9f856d15afa4077848276eaf432c2aa` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.414317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f856d15afa4077848276eaf432c2aa)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.416304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.464979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.476965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f856d15afa4077848276eaf432c2aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:27.485209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f856d15afa4077848276eaf432c2aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:27.501322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f856d15afa4077848276eaf432c2aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:27.502074 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f856d15afa4077848276eaf432c2aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:27.502678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f856d15afa4077848276eaf432c2aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:27.517554 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1266/2538] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:27 2022] DELETE /identity/v3/users/e9f856d15afa4077848276eaf432c2aa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:27.522246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.522550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.522830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.541790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:27.543025 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:27.616794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1273/2539] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:27.627855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.629294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:27.704942 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9688e-7704-4330-828d-9d128646fa18 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.711940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.713104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.713675 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.714250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.733576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.733576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.741149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1267/2540] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:28.043557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.045277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.059838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e95c7d-b46e-4121-9a49-eb7640c0431c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.064815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.065718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.066032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.066313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] PATH_INFO: `/v3/projects/4bf159f67d8f44209d36afb332f8f6e1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.067564 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf159f67d8f44209d36afb332f8f6e1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.069765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.095123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.098335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.138745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf159f67d8f44209d36afb332f8f6e1', 'request_id': 'req-069a569a-ab86-4d39-a2e5-5f3e09ab0366'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.146551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf159f67d8f44209d36afb332f8f6e1', 'request_id': 'req-069a569a-ab86-4d39-a2e5-5f3e09ab0366'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.154698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf159f67d8f44209d36afb332f8f6e1', 'request_id': 'req-069a569a-ab86-4d39-a2e5-5f3e09ab0366'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.172045 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] The token cache is being invalidate because project 4bf159f67d8f44209d36afb332f8f6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:28.172428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.173170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.175424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1274/2541] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:28 2022] DELETE /identity/v3/projects/4bf159f67d8f44209d36afb332f8f6e1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:28.179255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.180811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.181654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba204-f6b4-4b5d-9121-4c200412780d service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.238727 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.240179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.253813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.254492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.254776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.255208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.257590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.264582 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a569a-ab86-4d39-a2e5-5f3e09ab0366 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.269833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.270802 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.270944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.271053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.271327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] PATH_INFO: `/v3/users/be189bc20706425fadb5d4dc4dda6e99` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.272607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.272798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be189bc20706425fadb5d4dc4dda6e99)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.274637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.283678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1268/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:28.334262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be189bc20706425fadb5d4dc4dda6e99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.335037 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be189bc20706425fadb5d4dc4dda6e99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.335784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be189bc20706425fadb5d4dc4dda6e99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.344428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be189bc20706425fadb5d4dc4dda6e99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.353162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e38bb2e1-4275-4846-865a-b2d464cad074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be189bc20706425fadb5d4dc4dda6e99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.373448 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1275/2543] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:28 2022] DELETE /identity/v3/users/be189bc20706425fadb5d4dc4dda6e99 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:28.675781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.688457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4aeef-92fe-4006-a116-c1a741aa0dad service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.763976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.765790 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.766123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.766656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] PATH_INFO: `/v3/projects/a504bcf1545342b09b1d79cba5e06bb1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.768480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a504bcf1545342b09b1d79cba5e06bb1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.770432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.828469 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a504bcf1545342b09b1d79cba5e06bb1', 'request_id': 'req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.835237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a504bcf1545342b09b1d79cba5e06bb1', 'request_id': 'req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.847697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a504bcf1545342b09b1d79cba5e06bb1', 'request_id': 'req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.856847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] The token cache is being invalidate because project a504bcf1545342b09b1d79cba5e06bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:28.857143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.857803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-83182647-3ce3-4cb8-ab67-3e84dacbe8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:28.864407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1269/2544] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:28 2022] DELETE /identity/v3/projects/a504bcf1545342b09b1d79cba5e06bb1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:28.864714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.865046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.865332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.885921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:28.887175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.956430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1276/2545] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:28.964862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.965347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.965692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.987040 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:28.988446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.998084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1270/2546] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:29.003346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.017630 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d827b-0d1a-41a6-b0dd-79dbf2e2dc8e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.023388 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.024040 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.024299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.024605 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.025318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:29.025645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.027285 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.045134 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1277/2547] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.046302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.056370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f0b1-bf89-4df4-97e9-985bafa1b3de None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.062032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.062731 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.063012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.063508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.064255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.066316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.101767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1271/2548] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:29.107300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.119947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48046d82-4e05-4170-b0cf-d68917091579 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.183038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.201320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.202166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.202493 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.202894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.203459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.205535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.232617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1278/2549] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:29.233861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.246653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3f50e-7f4d-41d6-92d1-112ca0dd0db7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.252970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.253662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.254103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.254541 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.255353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.257655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.269359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1272/2550] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:29.274814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.286974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcfe9-1ef7-4c79-a48d-a3e1029d6226 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.292154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.292984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.293268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.293543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.294283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.295899 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265a2b-d1bf-47f6-8145-0b5d85862e36 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.311863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1279/2551] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:29.313459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.325900 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77899778-20ad-4b4f-a510-f92d24b7c28e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.331012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.331924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.332207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.332508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] PATH_INFO: `/v3/projects/b2da95f5c7f648d6a81b7fe9b31ffb97/users/74b01ae2eb7b49ab8212d0c0f9bede7b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.350209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2da95f5c7f648d6a81b7fe9b31ffb97, user_id=74b01ae2eb7b49ab8212d0c0f9bede7b, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.353059 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.371362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1273/2552] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:29 2022] PUT /identity/v3/projects/b2da95f5c7f648d6a81b7fe9b31ffb97/users/74b01ae2eb7b49ab8212d0c0f9bede7b/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:29.384811 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.385304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.385790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.387430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.389582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.406559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.421838 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] MFA Rules not processed for user `74b01ae2eb7b49ab8212d0c0f9bede7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:29.423199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.472149 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96629ab4-7904-4539-b91f-182a7e8df0c0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.477464 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.478388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.478674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.478953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] PATH_INFO: `/v3/users/459425a588ca4f8b97ca7e1f62bc19ef` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.480239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459425a588ca4f8b97ca7e1f62bc19ef)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.482249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.503394 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1280/2553] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:29.508006 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.520737 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fd720-8421-490d-b428-23c8b195d9a4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.526981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.527664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.527984 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.528231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.528988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.532212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.537363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459425a588ca4f8b97ca7e1f62bc19ef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:29.545757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459425a588ca4f8b97ca7e1f62bc19ef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:29.560066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459425a588ca4f8b97ca7e1f62bc19ef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:29.560802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459425a588ca4f8b97ca7e1f62bc19ef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:29.561440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-435013f4-f029-4013-a45c-6880f7603564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459425a588ca4f8b97ca7e1f62bc19ef'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:29.565364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1281/2554] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:29.566595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.569311 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.577054 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1274/2555] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:29 2022] DELETE /identity/v3/users/459425a588ca4f8b97ca7e1f62bc19ef => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:29.581641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.581934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.582208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.602093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:29.603784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.643800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc152c-dbaf-4660-9aff-01924d0f6320 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.668910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1275/2556] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:29.679322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.680943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.688502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.708620 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.709331 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.709616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.709884 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.710627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.712656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.722926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.740521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1282/2557] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:29.741741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.751535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ddda-c6a4-4c0d-a27c-6e6aea209e4c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.755336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570af5e-2536-439d-bd1a-e3d333d5db10 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.755602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.756278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.756705 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.756978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.761678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.762378 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.762655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.762928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.763943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.765632 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.770698 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.773844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.778334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1283/2558] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:29.782862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.786681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1276/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:29.795079 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a9220-0ad3-46b5-abe2-fdaa7e499d8d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.800595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.801519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.801800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.802371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] PATH_INFO: `/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b/users/762a746c4c8244dcaa4299f0c8daee0e/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.824646 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaee4b8828249beb3cc65ab37c76c9b, user_id=762a746c4c8244dcaa4299f0c8daee0e, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.830544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.857341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1284/2560] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:29 2022] PUT /identity/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b/users/762a746c4c8244dcaa4299f0c8daee0e/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:29.863524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.874397 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99877c7-5f3b-41e1-aa9e-2719ed7d1b1e service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.941552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.942249 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.942711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.942990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.943919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.945873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c606-508e-4156-85d1-663bbed5f159 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.960689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1277/2561] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:29.962005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.974010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f0c08-386a-400a-8787-ce7ed36e043f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.979743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.980684 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.980961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.981239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] PATH_INFO: `/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b/users/762a746c4c8244dcaa4299f0c8daee0e/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.983333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eaee4b8828249beb3cc65ab37c76c9b, user_id=762a746c4c8244dcaa4299f0c8daee0e, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.986065 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.000804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1285/2562] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:29 2022] PUT /identity/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b/users/762a746c4c8244dcaa4299f0c8daee0e/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:30.011684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.012125 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.012515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.041562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] MFA Rules not processed for user `762a746c4c8244dcaa4299f0c8daee0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:30.043186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.070524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.072510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.129038 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1278/2563] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:30.140641 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.142388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.147214 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a147bc-adf5-4956-a656-9e6127afd8c7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.152804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.153718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.153997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.154265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] PATH_INFO: `/v3/projects/da11f50dd0ce4418b722b5e544ccd0af` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.155578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da11f50dd0ce4418b722b5e544ccd0af)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.157648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.205718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21d6a-4380-4825-8a36-ce5b5989ce82 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.216683 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da11f50dd0ce4418b722b5e544ccd0af', 'request_id': 'req-d5985e9c-959b-44a2-977b-20adfee6222d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.216956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.224004 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da11f50dd0ce4418b722b5e544ccd0af', 'request_id': 'req-d5985e9c-959b-44a2-977b-20adfee6222d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.230649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da11f50dd0ce4418b722b5e544ccd0af', 'request_id': 'req-d5985e9c-959b-44a2-977b-20adfee6222d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.244369 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] The token cache is being invalidate because project da11f50dd0ce4418b722b5e544ccd0af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:30.244660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.245290 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.247199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1286/2564] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:30 2022] DELETE /identity/v3/projects/da11f50dd0ce4418b722b5e544ccd0af => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:30.251036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.252508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.263340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.263947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.264220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.264540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.266497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.266629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.322894 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985e9c-959b-44a2-977b-20adfee6222d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.329022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.330015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.330382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.330719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] PATH_INFO: `/v3/users/5c4b12e01d42417fbe74f77a6a2d4d3c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.332091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4b12e01d42417fbe74f77a6a2d4d3c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.334545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.334891 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.336358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.346056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1279/2565] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:30.392075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4b12e01d42417fbe74f77a6a2d4d3c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.392996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4b12e01d42417fbe74f77a6a2d4d3c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.393790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4b12e01d42417fbe74f77a6a2d4d3c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.403042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4b12e01d42417fbe74f77a6a2d4d3c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.416664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4b12e01d42417fbe74f77a6a2d4d3c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.422517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.424418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.436382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef3658-a729-4106-84d7-bb4bd1980777 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.437315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1287/2566] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:30 2022] DELETE /identity/v3/users/5c4b12e01d42417fbe74f77a6a2d4d3c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:30.488242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.489064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.489597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.489881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.492086 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.556453 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.558261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.568601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1280/2567] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:30.732942 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.734599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.830074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852117b-2972-41e1-ad46-811d99329d12 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.836057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.837241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.837555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.837918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] PATH_INFO: `/v3/projects/b35e6545272a45628aa021d620c1929a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.839474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35e6545272a45628aa021d620c1929a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.841680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.870921 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.872763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.886545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28675945-dc4e-4703-b249-c20fe6db3a7a service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.887205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.902565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.902881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35e6545272a45628aa021d620c1929a', 'request_id': 'req-88548ce2-b176-4931-a527-e7ac8d316396'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.909651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35e6545272a45628aa021d620c1929a', 'request_id': 'req-88548ce2-b176-4931-a527-e7ac8d316396'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.916660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35e6545272a45628aa021d620c1929a', 'request_id': 'req-88548ce2-b176-4931-a527-e7ac8d316396'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.930238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] The token cache is being invalidate because project b35e6545272a45628aa021d620c1929a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:30.930524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.936232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88548ce2-b176-4931-a527-e7ac8d316396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:30.938373 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1288/2568] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:30 2022] DELETE /identity/v3/projects/b35e6545272a45628aa021d620c1929a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:30.943943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ecc-8186-4833-b1e5-cadf952dbcd3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.944256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ecc-8186-4833-b1e5-cadf952dbcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.944548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ecc-8186-4833-b1e5-cadf952dbcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.953998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.954915 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.955193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.955478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] PATH_INFO: `/v3/users/74b01ae2eb7b49ab8212d0c0f9bede7b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.956748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b01ae2eb7b49ab8212d0c0f9bede7b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.959104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.964606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fc022ecc-8186-4833-b1e5-cadf952dbcd3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:30.965844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc022ecc-8186-4833-b1e5-cadf952dbcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.974518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1289/2569] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:30.982677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.983058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.983343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.003229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:31.004489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.012706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1290/2570] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:31.014596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b01ae2eb7b49ab8212d0c0f9bede7b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.017093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.023934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b01ae2eb7b49ab8212d0c0f9bede7b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.027427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc86b52-aae8-46d4-afb5-0608aae8ae55 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.033092 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.033786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.034082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.034401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.035209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:31.035510 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.038171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.040549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b01ae2eb7b49ab8212d0c0f9bede7b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.041482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b01ae2eb7b49ab8212d0c0f9bede7b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.042392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-68539bde-310f-4f65-83a8-c88728d1d1ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b01ae2eb7b49ab8212d0c0f9bede7b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.054629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1291/2571] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:31.059669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.061805 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.066861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1281/2572] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:30 2022] DELETE /identity/v3/users/74b01ae2eb7b49ab8212d0c0f9bede7b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:31.071718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.072176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.072632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.077356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85b6bd-6a0a-41ec-bfbf-b3ce9abaced4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.094261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:31.095593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.140897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.141573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.141886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.142168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.142864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.145001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.164288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1282/2573] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:31.176794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1292/2574] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:31.177083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.177083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.178813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.222047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.239875 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addff749-5ea3-4098-b11a-ac00fe145b23 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.244433 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ef61f-4fdf-4f9e-8693-1a6978a80280 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.249965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.250636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.250917 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.251192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.251893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.253897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.276187 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1293/2575] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:31.280469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.291171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d821c0-9c64-43a3-a5c2-030bf18bf4b7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.293531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.294234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.294495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.294774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.296628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.297324 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.297606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.297881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.298613 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.300220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.304526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.305971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.306764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.313337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1294/2576] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:31.316746 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1283/2577] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:31.321347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.331719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb9bc-1e9c-46aa-afd1-ec86ca793d20 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.336875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.337557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.337843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.338124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.338861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.340548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9def660-11be-43ad-9a2e-b0222f6725bf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.352204 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1295/2578] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:31.355989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.366271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6899c5f-a711-4a26-bfd3-2bba70f99f21 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.371177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.372076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.372445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.372672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] PATH_INFO: `/v3/projects/236f856a2de54ea5853d06e262c39329/users/a4a0a50ab3c7481ab0339bae146fbc07/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.389994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236f856a2de54ea5853d06e262c39329, user_id=a4a0a50ab3c7481ab0339bae146fbc07, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.392772 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.407929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1284/2579] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:31 2022] PUT /identity/v3/projects/236f856a2de54ea5853d06e262c39329/users/a4a0a50ab3c7481ab0339bae146fbc07/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:31.418592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.418885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.419354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.453078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] MFA Rules not processed for user `a4a0a50ab3c7481ab0339bae146fbc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:31.454303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.524828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1296/2580] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:31.529522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.595997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f07d0-da9b-4d20-9f86-2922b1fc02a5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.601735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.602422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.602712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.602992 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.603710 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.605723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.628196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.630074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.641179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1285/2581] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:31.642053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.644453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb732a-cd90-417c-b630-01f1756b8722 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.704741 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.705628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.705909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.706184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] PATH_INFO: `/v3/projects/b2da95f5c7f648d6a81b7fe9b31ffb97` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.707457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2da95f5c7f648d6a81b7fe9b31ffb97)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.708467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.709642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.712571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5899676-9d82-40fa-bca9-da8b477f952d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.718802 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.719627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.719916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.720197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.720979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.722951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.743752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1286/2582] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:31 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:31.747761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.758391 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fc752-d1c1-4140-a43f-22b4abdb2c9a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.761602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2da95f5c7f648d6a81b7fe9b31ffb97', 'request_id': 'req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.767071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2da95f5c7f648d6a81b7fe9b31ffb97', 'request_id': 'req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.773382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2da95f5c7f648d6a81b7fe9b31ffb97', 'request_id': 'req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.787863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] The token cache is being invalidate because project b2da95f5c7f648d6a81b7fe9b31ffb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:31.788153 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.788980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.790659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1297/2583] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:31 2022] DELETE /identity/v3/projects/b2da95f5c7f648d6a81b7fe9b31ffb97 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:31.794806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.796202 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.808579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.822144 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.822912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.823297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.823560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.824400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.826029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.837096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1287/2584] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:31.841253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.847482 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.856302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b7c88-bcda-48f7-9404-84ea1058ad90 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.866604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.867106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.867390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.867680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] PATH_INFO: `/v3/projects/b310a7f493ee473ab3149f11f442537e/users/c6af0e7e9d8a4b93b84b47e7968ddb5c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.875602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429edcc-cfc6-41aa-ae98-4ff616a5ddec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.880846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.881734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.882012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.882286 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] PATH_INFO: `/v3/users/762a746c4c8244dcaa4299f0c8daee0e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.883496 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762a746c4c8244dcaa4299f0c8daee0e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.884897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b310a7f493ee473ab3149f11f442537e, user_id=c6af0e7e9d8a4b93b84b47e7968ddb5c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.885524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.887559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.903427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1288/2585] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:31 2022] PUT /identity/v3/projects/b310a7f493ee473ab3149f11f442537e/users/c6af0e7e9d8a4b93b84b47e7968ddb5c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:31.907439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.942140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762a746c4c8244dcaa4299f0c8daee0e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.943148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762a746c4c8244dcaa4299f0c8daee0e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.943963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762a746c4c8244dcaa4299f0c8daee0e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.951171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762a746c4c8244dcaa4299f0c8daee0e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.959785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762a746c4c8244dcaa4299f0c8daee0e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:31.973309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab29c00-1765-44a3-bf27-b69238c3f171 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.977682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1298/2586] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:31 2022] DELETE /identity/v3/users/762a746c4c8244dcaa4299f0c8daee0e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:31.978390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.979143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.979519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.979866 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.980827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.982626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b8ea-c5c4-44ec-95ba-35de90f1e1f1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.996657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1289/2587] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:31.997828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.999415 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.073712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274a11f-2ad3-484c-825a-3002e051b045 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.078699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.079619 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.079940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.080173 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] PATH_INFO: `/v3/projects/b310a7f493ee473ab3149f11f442537e/users/c6af0e7e9d8a4b93b84b47e7968ddb5c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.082454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b310a7f493ee473ab3149f11f442537e, user_id=c6af0e7e9d8a4b93b84b47e7968ddb5c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.085192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.097115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1299/2588] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:31 2022] PUT /identity/v3/projects/b310a7f493ee473ab3149f11f442537e/users/c6af0e7e9d8a4b93b84b47e7968ddb5c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:32.109362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.109362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.109778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.144369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] MFA Rules not processed for user `c6af0e7e9d8a4b93b84b47e7968ddb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.144369 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.202644 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.232197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1290/2589] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:32.244119 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.245992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.277000 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.284091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.285567 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.315695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b3088-c5a4-4a20-b6c8-951b4b6570b3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.320203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.320934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.321305 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.321664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.333506 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.335177 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.345053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1300/2590] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:32.363412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f30e17-841e-490a-b6f1-ed789f272ab0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.368976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.369897 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.370304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.370582 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] PATH_INFO: `/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.371771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eaee4b8828249beb3cc65ab37c76c9b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.373720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.421895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.429371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eaee4b8828249beb3cc65ab37c76c9b', 'request_id': 'req-a8af990a-5315-463b-9e5f-5feea68a069c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:32.433573 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7be0-0a1e-4633-a512-d415ec42cb39 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.436239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eaee4b8828249beb3cc65ab37c76c9b', 'request_id': 'req-a8af990a-5315-463b-9e5f-5feea68a069c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:32.445876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eaee4b8828249beb3cc65ab37c76c9b', 'request_id': 'req-a8af990a-5315-463b-9e5f-5feea68a069c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:32.456878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] The token cache is being invalidate because project 3eaee4b8828249beb3cc65ab37c76c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:32.457181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:32.457829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8af990a-5315-463b-9e5f-5feea68a069c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:32.459618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1291/2591] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:32 2022] DELETE /identity/v3/projects/3eaee4b8828249beb3cc65ab37c76c9b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:32.464569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297951b1-0478-4e51-b92f-fbd25c6f4bfb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.464878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297951b1-0478-4e51-b92f-fbd25c6f4bfb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.465136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297951b1-0478-4e51-b92f-fbd25c6f4bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.483198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-297951b1-0478-4e51-b92f-fbd25c6f4bfb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.484474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-297951b1-0478-4e51-b92f-fbd25c6f4bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.486576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.487226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.487505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.487779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.489789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.547601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1292/2592] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:32.551620 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.553939 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.557474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.557885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.558302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.564178 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1301/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:32.578980 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.580207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.589155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1293/2594] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:32.593655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.605919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4f20-a4ce-4be8-acb9-7ab3ee15187e service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.611383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.612069 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.612390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.612674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.613451 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:32.613764 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.615390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.627578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1302/2595] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:32.632876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.638928 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.643342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb33609-adc4-47e9-b416-7d66595aab70 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.649728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.650896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.651292 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.651733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.652524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.654858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.685165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1294/2596] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:32.689622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.701302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ea6b-2860-4bed-8db0-b9f7eca332da admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.768775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.769467 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.769754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.770191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.770750 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.772842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.782893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.794625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1303/2597] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:32.798702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.822496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8eb3-9199-4752-b8f7-71924afedf87 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.827219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.827904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.828185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.828517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.829256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.830853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.843002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1295/2598] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:32.847076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.857674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c41af-61ef-48d2-8fac-636ca256a423 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.867599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.868494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.868785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.869057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.869869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.871532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69dbb4-08b3-44e2-886c-4d99e7ac371e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.885810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1304/2599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:32.887775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.899290 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c0c4-3bc2-4ebc-924e-961a5b31a28f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.904504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.905574 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.905917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.906136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] PATH_INFO: `/v3/projects/a905849e09b748aa84645a027548ebad/users/4a428981720445da93dcee89d4fac7d5/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.922123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a905849e09b748aa84645a027548ebad, user_id=4a428981720445da93dcee89d4fac7d5, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.924968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.938989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1296/2600] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:32 2022] PUT /identity/v3/projects/a905849e09b748aa84645a027548ebad/users/4a428981720445da93dcee89d4fac7d5/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:32.950053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.950341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.950624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.977248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] MFA Rules not processed for user `4a428981720445da93dcee89d4fac7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.978483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.045652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1305/2601] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:33.058007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.059905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.120094 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f7567-2460-4b64-a47e-1f8b34fbc1ac admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.124650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.125363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.125665 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.125948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.135368 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.136882 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ddc0c-7d6b-4b56-a489-5af2bd084648 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.148104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1297/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:33.169604 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.236211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c28e8f-cb7e-4cad-b902-869e8e393e6a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.241301 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.242185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.242470 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.242740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] PATH_INFO: `/v3/users/4a428981720445da93dcee89d4fac7d5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.243969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a428981720445da93dcee89d4fac7d5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.245982 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.297889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a428981720445da93dcee89d4fac7d5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.298606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a428981720445da93dcee89d4fac7d5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.299267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a428981720445da93dcee89d4fac7d5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.308137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a428981720445da93dcee89d4fac7d5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.316217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a428981720445da93dcee89d4fac7d5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.333064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.339479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1306/2603] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:33 2022] DELETE /identity/v3/users/4a428981720445da93dcee89d4fac7d5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:33.344525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.345166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.345166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.365247 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:33.366501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.437635 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1298/2604] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:33.448261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.449783 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.516409 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28419c92-e45d-40ae-b119-ce4c5b23ddb9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.520112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.521023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.521313 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.521595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.531618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.533296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64467189-e30b-4e82-87a3-2a6c7f92dc30 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.544279 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1307/2605] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:33.845622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.847676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.864834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606aeba-d63e-44e4-817c-4f087a8d635c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.875470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.876312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.876617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.876893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] PATH_INFO: `/v3/projects/a905849e09b748aa84645a027548ebad` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.878764 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a905849e09b748aa84645a027548ebad)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.880817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.933773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.937571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a905849e09b748aa84645a027548ebad', 'request_id': 'req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.946786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a905849e09b748aa84645a027548ebad', 'request_id': 'req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.955115 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a905849e09b748aa84645a027548ebad', 'request_id': 'req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.970319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] The token cache is being invalidate because project a905849e09b748aa84645a027548ebad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:33.970609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.971289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b31a51f2-c3ff-4b92-bff9-5815a60fd7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:33.972991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1299/2606] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:33 2022] DELETE /identity/v3/projects/a905849e09b748aa84645a027548ebad => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:33.978183 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.978472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.978756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.001443 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:34.002656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:34.071401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1308/2607] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:34.080271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.080793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.080991 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.106148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:34.107394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:34.117563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1300/2608] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:34.122024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.136367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41a2e5-7735-4e24-b76c-746944dcd703 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.141948 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.142656 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.142937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.143221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.143971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:34.144282 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.146042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.166488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1309/2609] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:34.168073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.179017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455d50-84cd-49a3-a849-371651b04b06 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.185175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.185862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.186154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.186428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.187138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.189123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.222809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1301/2610] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:34 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:34.227297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.238674 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca769-8ab8-4be2-8b01-61e3cca9328d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.302871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.303670 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.303974 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.304246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.305068 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.308038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.331275 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1310/2611] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:34 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:34.335740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.346787 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d46080-1602-4823-88f2-6345b054aeef admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.351783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.352487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.352774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.353050 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.353817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.355425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.366362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1302/2612] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:34.370791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.381696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749d77-6c9c-4dab-bd44-2b88eb000741 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.387642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.388240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.388559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.388833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.389563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.391168 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce03890-a5cf-415f-93e9-cfca9f92da22 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.402620 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1311/2613] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:34.407011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.417957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb6aab-c201-4329-89c9-c4e68519fd1f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.422661 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.423568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.423853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.424129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] PATH_INFO: `/v3/projects/d85251a4002041efad20c7a1adfcae7d/users/d68e3c40c0224f23a4160e247666d8f3/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.441192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85251a4002041efad20c7a1adfcae7d, user_id=d68e3c40c0224f23a4160e247666d8f3, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.444349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.460306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1303/2614] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:34 2022] PUT /identity/v3/projects/d85251a4002041efad20c7a1adfcae7d/users/d68e3c40c0224f23a4160e247666d8f3/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:34.472164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.472487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.472772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.479208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.505718 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] MFA Rules not processed for user `d68e3c40c0224f23a4160e247666d8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:34.506961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:34.577150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1312/2615] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:34.589696 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.591380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:34.657812 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b685b6-6bd8-483c-80ba-f03ad12f3753 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.662099 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.662617 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.662902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.663180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.674382 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.675835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6190c1-f207-4fc9-8a88-4038362341fb service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.685985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1304/2616] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:36.717172 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.718790 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:36.798333 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227181d-6983-4743-8b33-48880d7cdbca None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.803531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.804464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.804744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.805010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] PATH_INFO: `/v3/users/a4a0a50ab3c7481ab0339bae146fbc07` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.806248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a0a50ab3c7481ab0339bae146fbc07)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.808593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.866382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a0a50ab3c7481ab0339bae146fbc07'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:36.867480 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a0a50ab3c7481ab0339bae146fbc07'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:36.868123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a0a50ab3c7481ab0339bae146fbc07'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:36.876508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a0a50ab3c7481ab0339bae146fbc07'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:36.886112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a0a50ab3c7481ab0339bae146fbc07'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:36.907538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1313/2617] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:36 2022] DELETE /identity/v3/users/a4a0a50ab3c7481ab0339bae146fbc07 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:36.912825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.913435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.913435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.934444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:36.936910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:37.006425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1305/2618] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:37.016919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:37.019137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:37.035280 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:37.082174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f5114-ea4c-435a-a03d-e64713b95191 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:37.086398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:37.087075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:37.087359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:37.087631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:37.097196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:37.098652 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:37.108929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1314/2619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:37.409389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:37.411589 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:37.432711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3732933-ba2b-4ccc-bdce-a69b3d72e85e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:37.438088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:37.439132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:37.439488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:37.439777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] PATH_INFO: `/v3/projects/236f856a2de54ea5853d06e262c39329` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:37.441363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=236f856a2de54ea5853d06e262c39329)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:37.444552 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:37.454655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:37.503197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236f856a2de54ea5853d06e262c39329', 'request_id': 'req-452db778-02af-4ffc-af15-5ed54533a29f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.510656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236f856a2de54ea5853d06e262c39329', 'request_id': 'req-452db778-02af-4ffc-af15-5ed54533a29f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.520228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236f856a2de54ea5853d06e262c39329', 'request_id': 'req-452db778-02af-4ffc-af15-5ed54533a29f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.531637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] The token cache is being invalidate because project 236f856a2de54ea5853d06e262c39329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:37.531927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.532711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.534258 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1306/2620] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:37 2022] DELETE /identity/v3/projects/236f856a2de54ea5853d06e262c39329 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:37.538882 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:37.549548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3ab50-cd1a-4757-b479-66d6dc86689f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:37.625724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:37.626645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:37.626931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:37.627232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] PATH_INFO: `/v3/users/c6af0e7e9d8a4b93b84b47e7968ddb5c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:37.628566 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6af0e7e9d8a4b93b84b47e7968ddb5c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:37.630757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:37.682663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6af0e7e9d8a4b93b84b47e7968ddb5c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.683459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6af0e7e9d8a4b93b84b47e7968ddb5c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.684289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6af0e7e9d8a4b93b84b47e7968ddb5c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.694681 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6af0e7e9d8a4b93b84b47e7968ddb5c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.706708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b4a2bd5c-5587-4eb5-a69c-c11bdea70bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6af0e7e9d8a4b93b84b47e7968ddb5c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.725295 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1315/2621] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:37 2022] DELETE /identity/v3/users/c6af0e7e9d8a4b93b84b47e7968ddb5c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:37.775366 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:37.777011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:37.791623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452db778-02af-4ffc-af15-5ed54533a29f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:37.857020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:37.858032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:37.858412 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:37.858749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] PATH_INFO: `/v3/users/d68e3c40c0224f23a4160e247666d8f3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:37.860171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68e3c40c0224f23a4160e247666d8f3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:37.862206 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:37.914776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68e3c40c0224f23a4160e247666d8f3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.921997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68e3c40c0224f23a4160e247666d8f3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.935528 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68e3c40c0224f23a4160e247666d8f3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.936214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68e3c40c0224f23a4160e247666d8f3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.936904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68e3c40c0224f23a4160e247666d8f3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:37.951867 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1307/2622] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:37 2022] DELETE /identity/v3/users/d68e3c40c0224f23a4160e247666d8f3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:37.956598 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:37.956868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:37.957150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:37.974417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:37.975749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.007691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:38.042117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.044597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.058556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1316/2623] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:38.064606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53a384-3bc2-485a-a3a9-c1da750ebe68 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.069019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.071729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.072244 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.073367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.073785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.074187 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] PATH_INFO: `/v3/projects/b310a7f493ee473ab3149f11f442537e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.075593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b310a7f493ee473ab3149f11f442537e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.078134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.093964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:38.139434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b310a7f493ee473ab3149f11f442537e', 'request_id': 'req-7f7996b2-45b6-477b-8496-a64211bdb702'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.143356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d89031-16fd-4a23-ab0b-c04e2d245814 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.146728 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b310a7f493ee473ab3149f11f442537e', 'request_id': 'req-7f7996b2-45b6-477b-8496-a64211bdb702'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.156387 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b310a7f493ee473ab3149f11f442537e', 'request_id': 'req-7f7996b2-45b6-477b-8496-a64211bdb702'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.172773 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] The token cache is being invalidate because project b310a7f493ee473ab3149f11f442537e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:38.173072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.173852 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7f7996b2-45b6-477b-8496-a64211bdb702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.178857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1308/2624] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:38 2022] DELETE /identity/v3/projects/b310a7f493ee473ab3149f11f442537e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:38.181076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f4930-38ea-470a-8f36-b342d8171ba2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.181371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f4930-38ea-470a-8f36-b342d8171ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.181653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f4930-38ea-470a-8f36-b342d8171ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.207626 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.208346 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.208732 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.209266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-823f4930-38ea-470a-8f36-b342d8171ba2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:38.209801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.210445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-823f4930-38ea-470a-8f36-b342d8171ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.212464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.281835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1309/2625] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:38.285536 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.287413 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.289874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.290168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.290451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.299535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1317/2626] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:38.311457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:38.312723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.322002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1310/2627] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:38.327086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.339833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a13f-0d72-4b3c-b5b4-4e8d40d352ac service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.345110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.345812 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.346088 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.346371 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.347302 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:38.347589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.349461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.371504 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1318/2628] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:38.371844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.383957 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f038248-8963-4eb1-b4db-80800832f439 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.389548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.390339 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.390813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.391097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.391814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.393778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.423669 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1311/2629] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:38.428347 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.439141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7732-85fc-448b-a2b8-f37d2f95e31e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.467648 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:38.507094 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.508677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.508677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.508677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.510201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.511421 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.538266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1319/2630] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:38.539879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.551304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49a8f-5d38-4da2-b214-0c0d78b39e0e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.557974 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.558666 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.558948 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.559221 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.559969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.561627 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.573226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1312/2631] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:38.578046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.588930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2345f-b21f-46f3-8199-fa5d1daa03a1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.597399 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.598133 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.598667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.599001 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.599902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.602188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.616428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.619834 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1320/2632] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:38.621071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.622281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.635438 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0f61-8891-4df4-9967-e9aff724fd59 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.638638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ca923-b698-4e5b-bac1-6e3a014c4bba admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.642265 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.643281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.643565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.643753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.643879 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] PATH_INFO: `/v3/projects/c56d524bcaff40d1b8ac480069394a1e/users/22481c10209d409786aeb51937afa85a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.644676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.644959 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.645252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] PATH_INFO: `/v3/projects/d85251a4002041efad20c7a1adfcae7d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.646550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85251a4002041efad20c7a1adfcae7d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.648870 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.665226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56d524bcaff40d1b8ac480069394a1e, user_id=22481c10209d409786aeb51937afa85a, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.667928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4b83c-fa40-40c6-95ea-4a34658b9dfd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.683035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1321/2633] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:38 2022] PUT /identity/v3/projects/c56d524bcaff40d1b8ac480069394a1e/users/22481c10209d409786aeb51937afa85a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:38.695234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.695708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.695880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.703893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85251a4002041efad20c7a1adfcae7d', 'request_id': 'req-75378fa3-4176-437a-8aab-4293f0adf348'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.710478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85251a4002041efad20c7a1adfcae7d', 'request_id': 'req-75378fa3-4176-437a-8aab-4293f0adf348'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.712279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:38.720275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85251a4002041efad20c7a1adfcae7d', 'request_id': 'req-75378fa3-4176-437a-8aab-4293f0adf348'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.725897 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] MFA Rules not processed for user `22481c10209d409786aeb51937afa85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:38.727164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.734212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] The token cache is being invalidate because project d85251a4002041efad20c7a1adfcae7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:38.734525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.735203 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75378fa3-4176-437a-8aab-4293f0adf348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:38.736862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1313/2634] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:38 2022] DELETE /identity/v3/projects/d85251a4002041efad20c7a1adfcae7d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:38.741924 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7b52-0efc-44e7-a0d9-f08b485ebc06 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.742220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7b52-0efc-44e7-a0d9-f08b485ebc06 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.742516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b7b52-0efc-44e7-a0d9-f08b485ebc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.762133 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2c6b7b52-0efc-44e7-a0d9-f08b485ebc06 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:38.763655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2c6b7b52-0efc-44e7-a0d9-f08b485ebc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.798012 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1322/2635] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:38.802534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.804292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.836502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1314/2636] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:38.846975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.846975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.846975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.870939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:38.872181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:38.879444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bd15a-2573-4e35-97b4-1c93e640f243 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.880588 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1315/2637] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:38.884887 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.887831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.888579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.888890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.889171 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.889871 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.891817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.894055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bb3f7-4eb6-4a41-a2ca-18dab2b3a914 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.899335 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.900008 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.900374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.900656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.901396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:38.901707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.903444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.907366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:38.918154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1316/2638] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:38.920811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.922105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1323/2639] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:38.925774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:38.931264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98b083-43a7-451b-94cd-ae99fbc0a50a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.987614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458459d4-a0b9-4236-b2e5-55d9dbeaefe2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:38.988778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.989451 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.989736 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.990013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.990846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.992706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:38.992886 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:38.993693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:38.994036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:38.994283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:38.994988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:38.996957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.019550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1324/2640] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:39.023427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.025006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1317/2641] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:38 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:39.028898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.034195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c32b9-122d-4111-bc50-586913cd9a0c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.077307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.094483 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cc85b-dc0a-4d2b-a293-98bf51de35c0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.094911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.095725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.096010 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.096289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.097064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.098676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.099211 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.099911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.100200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.100508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.101204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.103121 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.109256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1325/2642] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:39.113127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.124029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77e0ce-2343-42a1-966a-1a8d220f83c8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.129430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1318/2643] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:39.129587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.129587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.129885 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.130160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] PATH_INFO: `/v3/projects/359a4d638c53421194ec7f78ca02394f/users/4be6632d471e443ebeddfcad8b18fa40/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.130281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.140549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4307c-d313-4162-bd1f-7e1151e8197a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.145308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.146000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.146275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.146693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.146829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] RBAC: Authorizing `identity:create_grant(project_id=359a4d638c53421194ec7f78ca02394f, user_id=4be6632d471e443ebeddfcad8b18fa40, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.147595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.149242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.149893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.160129 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1319/2644] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:39.164157 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.165757 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1326/2645] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:39 2022] PUT /identity/v3/projects/359a4d638c53421194ec7f78ca02394f/users/4be6632d471e443ebeddfcad8b18fa40/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:39.169425 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.174685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f830a-2da3-433a-9cd8-20a612015f44 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.234560 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.238668 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556a19-df0c-41a5-b7af-7cb91539542b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.239087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.239838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.240250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.240699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.241518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.243435 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.246259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.247380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.247664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.247938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.248716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.250498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.258291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1320/2646] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:39.259835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.264449 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1327/2647] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:39.265571 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.270672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b9fca-5686-4b1d-bf88-de2d1e657cad admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.276503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ed668-c2c2-4d28-849b-ddceb927948b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.276809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.277711 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.277994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.278271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] PATH_INFO: `/v3/projects/dc42ab5a1ffa4c339fe6f198ef976261/users/ece18005e0c34e02b09378a2514de612/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.282162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.282460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.282714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.283018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] PATH_INFO: `/v3/projects/359a4d638c53421194ec7f78ca02394f/users/4be6632d471e443ebeddfcad8b18fa40/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.285489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] RBAC: Authorizing `identity:create_grant(project_id=359a4d638c53421194ec7f78ca02394f, user_id=4be6632d471e443ebeddfcad8b18fa40, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.288087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31323f87-a77a-4793-8c06-6c9f575b556c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.300348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc42ab5a1ffa4c339fe6f198ef976261, user_id=ece18005e0c34e02b09378a2514de612, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.301015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1328/2648] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:39 2022] PUT /identity/v3/projects/359a4d638c53421194ec7f78ca02394f/users/4be6632d471e443ebeddfcad8b18fa40/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:39.303185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1ad39-c779-475c-a247-346e429fad66 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.312042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.312357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.312635 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.319265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1321/2649] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:39 2022] PUT /identity/v3/projects/dc42ab5a1ffa4c339fe6f198ef976261/users/ece18005e0c34e02b09378a2514de612/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:39.331489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.331780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.332056 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.343057 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] MFA Rules not processed for user `4be6632d471e443ebeddfcad8b18fa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.344270 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.360956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] MFA Rules not processed for user `ece18005e0c34e02b09378a2514de612`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.362230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.415833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1329/2650] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:39.426886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.428636 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.433633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1322/2651] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:39.437779 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.470388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.491287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117be6-7f0f-443f-87c5-9f0c7d9d8352 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.494756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.495416 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.495878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.496179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.503951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d12344-93b5-41a4-911c-810e814acb67 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.507266 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.508770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.509598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.510273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.510560 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.510840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.511529 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.513484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.518533 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1330/2652] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:39.548398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1323/2653] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:39.549600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.561937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62f48-e2a3-4e27-a8ad-f32ac0af57bd service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.629157 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.629839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.630128 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.630406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.631278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.633440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.650800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.661714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1331/2654] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:39.663406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.674976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3a054-c418-4327-b409-abf9bc4b1052 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.681020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.681699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.682033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.682614 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.683371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.685031 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.697519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1324/2655] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:39.704981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.712836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f560-3ce0-4031-ae2a-fae4844b2c9e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.718739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.719553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.719924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.720267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.721054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.722890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb2574-4d5f-48c8-a02f-6629377cf35f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.734798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1332/2656] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:39.738843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.749467 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b2390-b9a2-4ecf-8eff-46413ff8b13b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.754898 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.755827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.756111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.756422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] PATH_INFO: `/v3/projects/3bb60be77eaf40d1a0ee163f8f95428c/users/9f80ccfb79fa4b5d97c51546f4bf0b63/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.772933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb60be77eaf40d1a0ee163f8f95428c, user_id=9f80ccfb79fa4b5d97c51546f4bf0b63, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.775883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.791590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1325/2657] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:39 2022] PUT /identity/v3/projects/3bb60be77eaf40d1a0ee163f8f95428c/users/9f80ccfb79fa4b5d97c51546f4bf0b63/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:39.802760 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.803211 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.803519 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.837229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] MFA Rules not processed for user `9f80ccfb79fa4b5d97c51546f4bf0b63`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.838550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.906078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.929213 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1333/2658] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:39.936739 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.983758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265af1f5-def5-45ff-8eba-16be9f3130a5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.988892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.989798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.990078 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.990384 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] PATH_INFO: `/v3/users/22481c10209d409786aeb51937afa85a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.991859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22481c10209d409786aeb51937afa85a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.993903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.017675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363c616-4643-4ed5-8b76-5a99d261e071 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.023507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.024514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.024800 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.025067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.025793 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.027937 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.048688 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22481c10209d409786aeb51937afa85a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.056918 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22481c10209d409786aeb51937afa85a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.057714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1334/2659] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:40.061804 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.070315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22481c10209d409786aeb51937afa85a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.071156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22481c10209d409786aeb51937afa85a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.071785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2afaa95c-0bd2-4dba-9fb7-cef3dc5a49b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22481c10209d409786aeb51937afa85a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.082177 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.091738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1326/2660] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:39 2022] DELETE /identity/v3/users/22481c10209d409786aeb51937afa85a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:40.096842 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.097166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.097452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.115114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:40.116449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.131608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fade3-2729-4ce6-8707-9cdedc6e336e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.137376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.183216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1327/2661] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:40.193930 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.195602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.202735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.203424 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.203714 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.203987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.204719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.206682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.228616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1335/2662] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:40.234455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.236667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.250808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e4700-cb64-4aca-9ff2-666da4c43ef5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.261238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.261559 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.261862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.262140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.262890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.264631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.268752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978f3fa-6ef2-4a03-8e3b-c9b89e5ec999 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.274535 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.275219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.275508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.275850 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.279230 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1336/2663] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:40.284298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.291931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.293436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.295403 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ac74b-700d-48cc-b19f-9b95fb7767ca admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.300527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.301555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.301865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.302162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] PATH_INFO: `/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0/users/695aab9518814d5eb4b6d77e48e25294/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.305626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1328/2664] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:40.320623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77a6bb643f64aa68f02bc6ef597d6e0, user_id=695aab9518814d5eb4b6d77e48e25294, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.324261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.344182 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1337/2665] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:40 2022] PUT /identity/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0/users/695aab9518814d5eb4b6d77e48e25294/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:40.349336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.362166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03deff5-ac99-4739-9ab4-f9d9e1e011ac service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.440402 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.441166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.441395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.441679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.442424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.444502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3347e-15e4-4b55-a304-baa9f216fa75 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.457306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1329/2666] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:40.461569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.480454 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ddaf-9ead-4c8f-91c6-1d69289f6457 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.489825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.491402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.491402 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.491769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] PATH_INFO: `/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0/users/695aab9518814d5eb4b6d77e48e25294/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.494028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77a6bb643f64aa68f02bc6ef597d6e0, user_id=695aab9518814d5eb4b6d77e48e25294, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.496794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.510473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1338/2667] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:40 2022] PUT /identity/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0/users/695aab9518814d5eb4b6d77e48e25294/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:40.519919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.520565 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.520855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.552100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] MFA Rules not processed for user `695aab9518814d5eb4b6d77e48e25294`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:40.553548 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.591254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.647971 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.653621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1330/2668] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:40.654488 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.665510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.667459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.735593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc636-efa3-4ee1-801b-2bf4fb79c91f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.738869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2855d-c7d3-43ac-9a7a-c19fcd5808a2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.740797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.741952 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.742238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.742514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] PATH_INFO: `/v3/projects/c56d524bcaff40d1b8ac480069394a1e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.742678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.743336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.743625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.743907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.744515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56d524bcaff40d1b8ac480069394a1e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.745859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.746499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.795469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56d524bcaff40d1b8ac480069394a1e', 'request_id': 'req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.799553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.803954 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56d524bcaff40d1b8ac480069394a1e', 'request_id': 'req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.811332 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.812835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.812989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56d524bcaff40d1b8ac480069394a1e', 'request_id': 'req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.823797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1331/2669] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:40.827151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] The token cache is being invalidate because project c56d524bcaff40d1b8ac480069394a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:40.827445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.828080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:40.829963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1339/2670] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:40 2022] DELETE /identity/v3/projects/c56d524bcaff40d1b8ac480069394a1e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:40.833929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.848515 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b35779-a93d-4b8f-8bf9-dbf0e58506d4 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.916625 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.917522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.917801 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.918073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] PATH_INFO: `/v3/users/4be6632d471e443ebeddfcad8b18fa40` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.919310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be6632d471e443ebeddfcad8b18fa40)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.921313 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.947032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.996920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be6632d471e443ebeddfcad8b18fa40'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.011286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be6632d471e443ebeddfcad8b18fa40'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.025230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be6632d471e443ebeddfcad8b18fa40'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.025989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be6632d471e443ebeddfcad8b18fa40'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.026610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f3955728-a271-49fa-a80d-feeface030e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be6632d471e443ebeddfcad8b18fa40'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.041896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1332/2671] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:40 2022] DELETE /identity/v3/users/4be6632d471e443ebeddfcad8b18fa40 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:41.362841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.365359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:41.385958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a0ac-624a-4397-a4af-bc5ccc2ccbd1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.458689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.460365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.460658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.460935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] PATH_INFO: `/v3/projects/359a4d638c53421194ec7f78ca02394f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.462696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] RBAC: Authorizing `identity:delete_project(project_id=359a4d638c53421194ec7f78ca02394f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.465318 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.522762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359a4d638c53421194ec7f78ca02394f', 'request_id': 'req-c0470d42-4ae0-424d-a5f2-4d6a861c659c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.529116 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359a4d638c53421194ec7f78ca02394f', 'request_id': 'req-c0470d42-4ae0-424d-a5f2-4d6a861c659c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.535773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359a4d638c53421194ec7f78ca02394f', 'request_id': 'req-c0470d42-4ae0-424d-a5f2-4d6a861c659c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.551548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] The token cache is being invalidate because project 359a4d638c53421194ec7f78ca02394f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:41.551844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.552606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0470d42-4ae0-424d-a5f2-4d6a861c659c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:41.554699 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1340/2672] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:41 2022] DELETE /identity/v3/projects/359a4d638c53421194ec7f78ca02394f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:41.559965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.560264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.560572 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.583105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:41.584584 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:41.655969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1333/2673] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:41.665835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.666196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.666412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.694578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:41.695896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:41.706461 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1341/2674] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:41.711286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.722240 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75401e80-14be-47fa-99ca-f663afa3f5c9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.728670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.729596 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.729903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.730168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.730925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:41.731233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.733393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.748950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1334/2675] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:41.754821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.765844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9c6b1-3a91-4c7a-9cac-e0500606887a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.771447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.772377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.772849 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.773299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.774199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.776226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.810380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1342/2676] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:41.815272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.828020 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef486cc-e071-4c66-8334-054054c873a4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.898912 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.899667 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.900017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.900409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.901204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.903165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.935723 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1335/2677] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:41.939779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.951124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728400-0390-4085-89ec-06f0d61ba0d4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.955852 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.956562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.956847 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.957123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.957890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.959651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.971205 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1343/2678] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:41.975639 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.986846 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355791a9-002b-4ff4-9c30-14cca56be5f2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.992225 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.992951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.993233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.993504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.994241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.995858 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b6c4c-3bd8-4ece-b782-6c425afe179f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.007372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1336/2679] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:42.013577 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.027854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536d13-68d9-4e37-a705-3f43068fd01c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.033731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.034649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.034938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.035209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] PATH_INFO: `/v3/projects/0cb62c19d4df4ab583b9deaa5c141dc8/users/201d3268e44d435ba1dc7a86f026317f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.051972 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb62c19d4df4ab583b9deaa5c141dc8, user_id=201d3268e44d435ba1dc7a86f026317f, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.054993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.070446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1344/2680] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:42 2022] PUT /identity/v3/projects/0cb62c19d4df4ab583b9deaa5c141dc8/users/201d3268e44d435ba1dc7a86f026317f/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:42.082964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.083179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.083427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.091047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.117135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] MFA Rules not processed for user `201d3268e44d435ba1dc7a86f026317f`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:42.118568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.195109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1337/2681] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:42.200049 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.210707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.295148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fb2de-4936-4bf7-a2d6-dec8d430680f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.295148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.295148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.295148 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.295728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.295728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.299074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.344036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1345/2682] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:42 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:42.345150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.361513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcd8e5-c846-47af-b188-3afdb7d3e300 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.429334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.431497 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.432748 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.433060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.433457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.434263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.434404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.436281 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.457847 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1338/2683] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:42 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:42.462104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.474693 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672a43-c24b-4acf-9d0a-8f663ba532ff admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.481707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.482401 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.482878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.483120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.483964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.485958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.498622 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1339/2684] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:42.503245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.510701 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9123-5ecb-435a-bb41-e99d04f5dea8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.516070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.517038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.517486 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.517765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9bbc5a-9a1f-44de-bc10-1de53390db97 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.518087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.520627 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.524804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.526172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.526642 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.527120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] PATH_INFO: `/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df/users/8085ea40658e4c82adec338e4f7b4293/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.552334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2810fbaa8d5e4c2b8a374f6e3580b3df, user_id=8085ea40658e4c82adec338e4f7b4293, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.555784 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.578555 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1340/2685] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:42 2022] PUT /identity/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df/users/8085ea40658e4c82adec338e4f7b4293/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:42.587447 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.598391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.600625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.610846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1346/2686] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:42.642510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.659127 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487fd62-e0ad-4f87-86de-235ee603d000 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.672615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.673466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.673785 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.674055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.674813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.676510 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f510c-dfd9-45a9-be90-b896dba572b6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.692140 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1341/2687] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:42.693541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.706161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b31ad-58eb-4f34-9c16-9ebc7b7d6c48 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.712868 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.713900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.714243 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.714508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] PATH_INFO: `/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df/users/8085ea40658e4c82adec338e4f7b4293/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.717130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2810fbaa8d5e4c2b8a374f6e3580b3df, user_id=8085ea40658e4c82adec338e4f7b4293, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:42.719839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:42.734241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1347/2688] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:42 2022] PUT /identity/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df/users/8085ea40658e4c82adec338e4f7b4293/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:42.745273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.745598 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.745925 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.775457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] MFA Rules not processed for user `8085ea40658e4c82adec338e4f7b4293`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:42.776781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.857208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1342/2689] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:42.869141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.871195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.910999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.940946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49bd-f23f-41e7-af6b-37e365ac4e7d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.950075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.950748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.951026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.951298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.015401 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.016920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.027880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1348/2690] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:43.253534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.275306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fccb-7628-47cd-a286-49f9df5e2b98 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.283414 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.284655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.285145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.285624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.288621 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:43.393889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.397216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.415415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1343/2691] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:47.601474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.609394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.611168 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:47.612531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acd2f3-8e6d-44fc-afc3-37f8a2038957 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.617473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.618239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.618622 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.618869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.630650 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.633362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0de6d-5081-4dc7-b643-26ad5580fe2a service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.645594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1349/2692] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:47.700781 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7b88-be23-4a78-b097-05cc4ebbde6e service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.705459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.706378 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.706663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.706976 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] PATH_INFO: `/v3/users/ece18005e0c34e02b09378a2514de612` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.708385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece18005e0c34e02b09378a2514de612)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.710507 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.713190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.786459 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece18005e0c34e02b09378a2514de612'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:47.804663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece18005e0c34e02b09378a2514de612'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:47.830506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece18005e0c34e02b09378a2514de612'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:47.831911 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece18005e0c34e02b09378a2514de612'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:47.832894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece18005e0c34e02b09378a2514de612'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:47.871049 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1344/2693] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:47 2022] DELETE /identity/v3/users/ece18005e0c34e02b09378a2514de612 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:47.876167 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.876641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.876925 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.896860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:47.899014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:47.978052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1350/2694] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:47.986176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.988294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:48.062252 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4737c-d31a-49f6-9b0b-97df63461489 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.066516 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.067330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.067753 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.068186 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.078619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.080104 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.092195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1345/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:48.390628 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.392199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:48.404616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0589f54-b218-4055-b2e5-acd2342c8127 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.410130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.411024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.411303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.411580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] PATH_INFO: `/v3/projects/dc42ab5a1ffa4c339fe6f198ef976261` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.412997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc42ab5a1ffa4c339fe6f198ef976261)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.414959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.465284 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc42ab5a1ffa4c339fe6f198ef976261', 'request_id': 'req-b2e187da-592b-480e-b57a-394b10ae89f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.470698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc42ab5a1ffa4c339fe6f198ef976261', 'request_id': 'req-b2e187da-592b-480e-b57a-394b10ae89f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.476903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc42ab5a1ffa4c339fe6f198ef976261', 'request_id': 'req-b2e187da-592b-480e-b57a-394b10ae89f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.488466 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.495067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] The token cache is being invalidate because project dc42ab5a1ffa4c339fe6f198ef976261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:48.495067 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.496118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.498232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1351/2696] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:48 2022] DELETE /identity/v3/projects/dc42ab5a1ffa4c339fe6f198ef976261 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:48.502818 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.514511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f8863-852c-4772-b097-d99b5285547d service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.581407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.582300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.582580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.582860 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] PATH_INFO: `/v3/users/9f80ccfb79fa4b5d97c51546f4bf0b63` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.584051 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f80ccfb79fa4b5d97c51546f4bf0b63)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.586678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.616312 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.637128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f80ccfb79fa4b5d97c51546f4bf0b63'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.645014 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f80ccfb79fa4b5d97c51546f4bf0b63'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.657670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f80ccfb79fa4b5d97c51546f4bf0b63'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.658446 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f80ccfb79fa4b5d97c51546f4bf0b63'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.659084 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f80ccfb79fa4b5d97c51546f4bf0b63'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:48.674304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1346/2697] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:48 2022] DELETE /identity/v3/users/9f80ccfb79fa4b5d97c51546f4bf0b63 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:48.962881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.965099 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:48.985589 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e187da-592b-480e-b57a-394b10ae89f6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.062064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.063016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.063612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.063740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] PATH_INFO: `/v3/projects/3bb60be77eaf40d1a0ee163f8f95428c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.065294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb60be77eaf40d1a0ee163f8f95428c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.067176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.130601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb60be77eaf40d1a0ee163f8f95428c', 'request_id': 'req-53177ace-3390-4b88-b8c3-1a45b0b02c2c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.137731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb60be77eaf40d1a0ee163f8f95428c', 'request_id': 'req-53177ace-3390-4b88-b8c3-1a45b0b02c2c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.144544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb60be77eaf40d1a0ee163f8f95428c', 'request_id': 'req-53177ace-3390-4b88-b8c3-1a45b0b02c2c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.158878 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] The token cache is being invalidate because project 3bb60be77eaf40d1a0ee163f8f95428c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:49.159155 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.159855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.161499 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1352/2698] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:48 2022] DELETE /identity/v3/projects/3bb60be77eaf40d1a0ee163f8f95428c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:49.166048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.167643 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:49.180633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4b395-c5f0-48fa-a153-834d5b813716 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.246814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.248506 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.248800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.249406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] PATH_INFO: `/v3/users/695aab9518814d5eb4b6d77e48e25294` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.250934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=695aab9518814d5eb4b6d77e48e25294)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.253263 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.329884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695aab9518814d5eb4b6d77e48e25294'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.339463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695aab9518814d5eb4b6d77e48e25294'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.354947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695aab9518814d5eb4b6d77e48e25294'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.355896 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695aab9518814d5eb4b6d77e48e25294'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.356538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4f4d34d-b6da-4c24-ba09-46c0c7908c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695aab9518814d5eb4b6d77e48e25294'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.376301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1347/2699] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:49 2022] DELETE /identity/v3/users/695aab9518814d5eb4b6d77e48e25294 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:49.706359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.710363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:49.729267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53177ace-3390-4b88-b8c3-1a45b0b02c2c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.805228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.806028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.806364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.806642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] PATH_INFO: `/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.808234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77a6bb643f64aa68f02bc6ef597d6e0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.810247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.852011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.869080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77a6bb643f64aa68f02bc6ef597d6e0', 'request_id': 'req-06426f09-9431-4341-abf7-2e7223a797cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.878327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77a6bb643f64aa68f02bc6ef597d6e0', 'request_id': 'req-06426f09-9431-4341-abf7-2e7223a797cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.887557 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77a6bb643f64aa68f02bc6ef597d6e0', 'request_id': 'req-06426f09-9431-4341-abf7-2e7223a797cb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.909136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] The token cache is being invalidate because project d77a6bb643f64aa68f02bc6ef597d6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:49.911993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.911993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-06426f09-9431-4341-abf7-2e7223a797cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:49.911993 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1353/2700] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:49 2022] DELETE /identity/v3/projects/d77a6bb643f64aa68f02bc6ef597d6e0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:49.917449 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.918230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.918230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.939491 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:49.940752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:49.962952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.017720 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1348/2701] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:50.029712 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.030525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.030525 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.054176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:50.055479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:50.065709 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1354/2702] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:50.071062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.084808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f2831-72ed-4941-bf4a-2e0ce2d91170 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.092415 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.093342 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.093823 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.094326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.095553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:50.096048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.098072 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.113581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1349/2703] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:50.118585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.128727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9bed9-d1be-43f0-9a88-0ef8cead2b28 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.137658 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.138408 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.138694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.139061 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.139665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.141679 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.184498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1355/2704] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:50 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:50.189179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.200754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb8c20-9728-444d-80f2-ef60d7dd0f6e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.294147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.294147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.294147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.294147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.294607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.294607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.319041 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1350/2705] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:50 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:50.324086 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.339505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74fdb-01ce-4629-8ce0-4c4a5d7cacbc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.347444 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.348545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.348698 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.348862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.351377 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.351545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.364355 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1356/2706] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:34:50.371173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.385836 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10630-8a92-4a9d-a05f-873db3cae72c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.393204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.393989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.394337 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.394659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.395418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.400180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf9def-5500-46fa-9ded-93167dbfd2be admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.414734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1351/2707] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:50.419294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.430787 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701034bd-2f7d-4d85-98db-15c1e308d208 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.443549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.445162 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.445524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.445801 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] PATH_INFO: `/v3/projects/62db9432d77f4e24abb5f506fd8b7f4e/users/defc63ed7ff84a40af71a475db376580/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.466924 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62db9432d77f4e24abb5f506fd8b7f4e, user_id=defc63ed7ff84a40af71a475db376580, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.470460 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.487789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1357/2708] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:50 2022] PUT /identity/v3/projects/62db9432d77f4e24abb5f506fd8b7f4e/users/defc63ed7ff84a40af71a475db376580/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:50.499839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.500198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.500580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.536893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] MFA Rules not processed for user `defc63ed7ff84a40af71a475db376580`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:50.538837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:50.636786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1352/2709] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:50.651091 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.653114 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:50.724112 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65efb12-d587-4c23-a86a-3098a56d473f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.728821 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.730289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.730289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.730289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.740055 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.741745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.751457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1358/2710] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:51.519361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.520827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:51.550367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.599354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382900d-8cf5-4606-b5a4-043bb7a6e21a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.603307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.603966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.604243 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.604547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.606383 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:51.682573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.685561 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2146-3ebb-4a37-811e-0fcb59a3eb74 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.702310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1353/2711] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:34:52.707521 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.708999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:52.728387 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cdaeb-7e37-463f-a177-a31ea061bd85 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.733886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.735054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.735859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.735859 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] PATH_INFO: `/v3/users/201d3268e44d435ba1dc7a86f026317f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.736916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201d3268e44d435ba1dc7a86f026317f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.738938 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.800115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201d3268e44d435ba1dc7a86f026317f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:52.801269 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201d3268e44d435ba1dc7a86f026317f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:52.801902 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201d3268e44d435ba1dc7a86f026317f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:52.807947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.811478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201d3268e44d435ba1dc7a86f026317f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:52.824479 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201d3268e44d435ba1dc7a86f026317f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:52.850058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1359/2712] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:52 2022] DELETE /identity/v3/users/201d3268e44d435ba1dc7a86f026317f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:52.855275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.856375 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.856916 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.876707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:52.878166 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:52.946686 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1354/2713] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:52.953685 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.955241 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:53.015705 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f2395-ddc5-43ba-a8ad-063b992c1bc2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.019591 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.020411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.020791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.021070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.032527 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.034144 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.044986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1360/2714] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:53.659813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.661234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:53.693103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922227f1-ab41-4603-b9f0-11675e67edcf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.707989 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.709210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.709672 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.709997 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] PATH_INFO: `/v3/projects/0cb62c19d4df4ab583b9deaa5c141dc8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.713433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb62c19d4df4ab583b9deaa5c141dc8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.716012 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.790576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb62c19d4df4ab583b9deaa5c141dc8', 'request_id': 'req-3ab16d42-237d-4554-9ad4-cbf536eef502'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:53.800019 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb62c19d4df4ab583b9deaa5c141dc8', 'request_id': 'req-3ab16d42-237d-4554-9ad4-cbf536eef502'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:53.813304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb62c19d4df4ab583b9deaa5c141dc8', 'request_id': 'req-3ab16d42-237d-4554-9ad4-cbf536eef502'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:53.817674 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.836356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] The token cache is being invalidate because project 0cb62c19d4df4ab583b9deaa5c141dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:53.838063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:53.839395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:53.843272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1355/2715] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:53 2022] DELETE /identity/v3/projects/0cb62c19d4df4ab583b9deaa5c141dc8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:53.856249 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.868258 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff871d7-2471-4bc5-b8c9-6e68dde1a44c service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.966200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.967773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.967773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.968206 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] PATH_INFO: `/v3/users/8085ea40658e4c82adec338e4f7b4293` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.969901 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8085ea40658e4c82adec338e4f7b4293)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.972186 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.053368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8085ea40658e4c82adec338e4f7b4293'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.053368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8085ea40658e4c82adec338e4f7b4293'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.053368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8085ea40658e4c82adec338e4f7b4293'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.056363 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8085ea40658e4c82adec338e4f7b4293'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.066043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ee09712b-c369-4893-b749-0a63ca411ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8085ea40658e4c82adec338e4f7b4293'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.074756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.102930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1361/2716] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:53 2022] DELETE /identity/v3/users/8085ea40658e4c82adec338e4f7b4293 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:54.462754 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.464380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:54.484679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab16d42-237d-4554-9ad4-cbf536eef502 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.555815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.557480 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.558023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.558429 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] PATH_INFO: `/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.560740 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2810fbaa8d5e4c2b8a374f6e3580b3df)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.562800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.620045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2810fbaa8d5e4c2b8a374f6e3580b3df', 'request_id': 'req-98e90909-2920-4b72-b9da-0934947fa5b7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.633662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2810fbaa8d5e4c2b8a374f6e3580b3df', 'request_id': 'req-98e90909-2920-4b72-b9da-0934947fa5b7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.648452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2810fbaa8d5e4c2b8a374f6e3580b3df', 'request_id': 'req-98e90909-2920-4b72-b9da-0934947fa5b7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.666315 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] The token cache is being invalidate because project 2810fbaa8d5e4c2b8a374f6e3580b3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:54.666805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.667722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-98e90909-2920-4b72-b9da-0934947fa5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:54.670473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1356/2717] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:54 2022] DELETE /identity/v3/projects/2810fbaa8d5e4c2b8a374f6e3580b3df => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:54.676749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.677042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.677327 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.701826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:54.703296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:54.802551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1362/2718] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:54.812430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.812820 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.813256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.837543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:54.839021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:54.849091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1357/2719] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:54.853529 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.870070 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c283b4-1fb4-42fc-b94f-908ee0ae3709 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.875381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.876059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.876368 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.876651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.877395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:54.877706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.879336 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.892743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1363/2720] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:54.897217 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.907298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b70a2-9edd-4761-8b31-ec67f0ee6fb5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.912362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.914181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.914181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.914181 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.915293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.917314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.945884 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1358/2721] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:54 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:54.950562 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.961158 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324289-704b-4f41-8667-6e92b721d932 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.023518 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.024208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.024532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.024807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.025513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.027484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.057725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1364/2722] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:54 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:55.062214 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.094623 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1464676-7700-4391-9d88-c4739aef2e59 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.101216 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.102089 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.102434 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.102776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.103587 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.109137 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.125513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1359/2723] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:55.129880 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.142549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7404481-5851-46c4-a173-ecb1d5243468 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.151405 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.157916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.160833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.161136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.162278 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.164026 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d765e-6286-4f7b-b71f-e99813fcac0f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.180893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1365/2724] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:55.182502 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.197735 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b01bb9-6ddf-492d-83b6-0ccc6cf5e740 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.203130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.203863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.204156 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.204468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] PATH_INFO: `/v3/projects/7ef1f8afa02043918b4e76f9af38d47f/users/3b100ef05c594d18b3868c2d0cb4f175/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.233219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef1f8afa02043918b4e76f9af38d47f, user_id=3b100ef05c594d18b3868c2d0cb4f175, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.235876 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.256958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1360/2725] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:55 2022] PUT /identity/v3/projects/7ef1f8afa02043918b4e76f9af38d47f/users/3b100ef05c594d18b3868c2d0cb4f175/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:55.336412 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.336690 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.337015 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.346484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.349016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.376410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] MFA Rules not processed for user `3b100ef05c594d18b3868c2d0cb4f175`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:55.377835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.381388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.434579 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.439527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04784849-c8ef-4a23-897d-9442f553dab5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.444613 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.445524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.445815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.446219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] PATH_INFO: `/v3/users/23abc809ab644c1da0270a35ac8bab21` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.447470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23abc809ab644c1da0270a35ac8bab21)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.450105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.463179 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1366/2726] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:55.475251 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.477524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.512684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23abc809ab644c1da0270a35ac8bab21'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.524199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23abc809ab644c1da0270a35ac8bab21'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.540697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23abc809ab644c1da0270a35ac8bab21'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.541505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23abc809ab644c1da0270a35ac8bab21'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.542196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23abc809ab644c1da0270a35ac8bab21'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.548104 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411c93c-f17c-47aa-893b-200e5752b8b1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.554585 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.555288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.555551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.555827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.558141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.561471 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1361/2727] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:34:55 2022] DELETE /identity/v3/users/23abc809ab644c1da0270a35ac8bab21 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:55.571361 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.572892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a14c5-2479-401e-af91-9282993a287d service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.583322 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1367/2728] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:55.760893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.762579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.855371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d6273-fae3-4eda-92b4-2f8508d03212 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.860474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.861536 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.861904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.862409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] PATH_INFO: `/v3/projects/dff1983b1b3047378b88d87d54aa709d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.863765 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff1983b1b3047378b88d87d54aa709d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.866626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.919819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.936000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff1983b1b3047378b88d87d54aa709d', 'request_id': 'req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.949685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff1983b1b3047378b88d87d54aa709d', 'request_id': 'req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.953538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff1983b1b3047378b88d87d54aa709d', 'request_id': 'req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.967135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] The token cache is being invalidate because project dff1983b1b3047378b88d87d54aa709d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:34:55.967513 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.968251 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a0abaf6e-35e3-4074-bd52-5b963f8ac5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:34:55.969920 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1362/2729] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:34:55 2022] DELETE /identity/v3/projects/dff1983b1b3047378b88d87d54aa709d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:55.976185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.977095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.977309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.996621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:55.997846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:56.062647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1368/2730] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:56.071655 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.072108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.072212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.092929 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:56.094175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:56.103712 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1363/2731] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:56.108256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.119203 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa69-9457-4b47-b785-0d58f0284c71 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.125005 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.125653 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.125945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.126220 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.126943 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.127257 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.128975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.142383 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1369/2732] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:56.146954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.157005 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc3ca7-7526-4ecb-a804-172ce4e3778f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.165936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.166766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.167058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.167334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.168204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.170359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.200389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1364/2733] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:56.204727 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.216044 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db7ba2-dcd2-4289-8f9a-21a5b2d14753 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.289716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.290400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.290719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.290994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.291689 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.293929 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.316248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1370/2734] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/users => generated 279 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:56.321062 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.333063 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69775a3c-5844-4510-bb05-6a294df75e31 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.338462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.338830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.339114 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.339389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.340153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.341805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.356984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1365/2735] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:56.358319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.368891 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fc66-86f5-4008-a4ae-11a60b61a909 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.373642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.374315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.374601 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.374881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.375606 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.377259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72beec7-0739-4d15-a338-45ca94282d25 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.379837 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.391545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1371/2736] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:56.395551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.409093 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa616f23-9e4b-47e3-b3e0-1e428d740806 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.415551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.416699 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.417120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.417526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] PATH_INFO: `/v3/projects/f10b12f0ada647989e12a209aba03f17/users/0773d79385b648d28eb102333c65b748/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.435386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10b12f0ada647989e12a209aba03f17, user_id=0773d79385b648d28eb102333c65b748, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.438257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.455048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1366/2737] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:56 2022] PUT /identity/v3/projects/f10b12f0ada647989e12a209aba03f17/users/0773d79385b648d28eb102333c65b748/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:56.466791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.467169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.467512 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.499358 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] MFA Rules not processed for user `0773d79385b648d28eb102333c65b748`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:56.500614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:56.589831 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1372/2738] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:56.594198 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.673087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876eca5-61e3-4539-8a5b-97d4f90216f0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.678354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.679033 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.679318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.679599 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.680293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.682633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.713508 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1367/2739] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/projects => generated 357 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:56.718524 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.733081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8c7f2-aa51-40a5-a4ea-de0f8e3129a0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.799854 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.800586 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.800869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.801184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.801931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.804611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.827459 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1373/2740] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:56.832460 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.844878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f8c9e-4540-4814-bc45-d87e3fbd6e6b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.850594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.851427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.851777 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.852100 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.853111 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.855159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.867526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1368/2741] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:56.872107 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.882945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9705d4-b755-4430-8572-0dc8c55136ce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.888129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.889074 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.889356 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.889631 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] PATH_INFO: `/v3/projects/af2ede11f88a4b0aad71ba636582ba43/users/d5c70571317a477c9414a3e032bc74b5/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.908818 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2ede11f88a4b0aad71ba636582ba43, user_id=d5c70571317a477c9414a3e032bc74b5, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.911484 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d904a-d107-4577-ac90-5cd24f040b2d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.929599 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1374/2742] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:56 2022] PUT /identity/v3/projects/af2ede11f88a4b0aad71ba636582ba43/users/d5c70571317a477c9414a3e032bc74b5/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:56.937624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.950190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d8949-fc6e-4a11-8c6a-d947fbfee8fc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.009694 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.014908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e5df4-9480-424f-b4b1-b7e5f27c0e2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.014908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e5df4-9480-424f-b4b1-b7e5f27c0e2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.014908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e5df4-9480-424f-b4b1-b7e5f27c0e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.034329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-416e5df4-9480-424f-b4b1-b7e5f27c0e2f None None] MFA Rules not processed for user `3b100ef05c594d18b3868c2d0cb4f175`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:57.035247 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-416e5df4-9480-424f-b4b1-b7e5f27c0e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:57.048838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.048838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.048838 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.049603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.049763 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.051396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.063135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1369/2743] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:34:57.067511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.081176 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb4e7-58fd-47c5-8b3b-c5cfa8ef3022 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.087145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.088045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.088349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.088750 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] PATH_INFO: `/v3/projects/af2ede11f88a4b0aad71ba636582ba43/users/d5c70571317a477c9414a3e032bc74b5/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.090829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2ede11f88a4b0aad71ba636582ba43, user_id=d5c70571317a477c9414a3e032bc74b5, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.093575 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.097113 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.114095 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1375/2744] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:34:57.128409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1370/2745] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:34:57 2022] PUT /identity/v3/projects/af2ede11f88a4b0aad71ba636582ba43/users/d5c70571317a477c9414a3e032bc74b5/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:57.134825 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096273-c610-4514-83fb-8e0677200055 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.135309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096273-c610-4514-83fb-8e0677200055 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.135774 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30096273-c610-4514-83fb-8e0677200055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.190255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-30096273-c610-4514-83fb-8e0677200055 None None] MFA Rules not processed for user `d5c70571317a477c9414a3e032bc74b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:57.192103 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-30096273-c610-4514-83fb-8e0677200055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:57.309400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1376/2746] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:34:57.327207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.329885 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:57.430807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc02ad0-859d-4d79-bfcc-d5d99264fa63 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.436797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.437998 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.438326 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.438733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.452556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.454327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.468739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1371/2747] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:34:57.592250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096273-c610-4514-83fb-8e0677200055 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.595417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-30096273-c610-4514-83fb-8e0677200055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:57.626908 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30096273-c610-4514-83fb-8e0677200055 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.637862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.639375 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.640130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.640889 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.786733 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.788372 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3ecab-0e48-4dc2-9db4-f171bf9fe667 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.801694 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1377/2748] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:07.461266 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:07.795600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=64 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:13.720047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.791361 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602a31-6db1-4eb8-a7f1-f2560b81f01b service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.796798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.797687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.797966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.798237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] PATH_INFO: `/v3/users/0773d79385b648d28eb102333c65b748` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.799492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0773d79385b648d28eb102333c65b748)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.801475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.857857 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0773d79385b648d28eb102333c65b748'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:13.865253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0773d79385b648d28eb102333c65b748'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:13.878868 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0773d79385b648d28eb102333c65b748'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:13.879685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0773d79385b648d28eb102333c65b748'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:13.880457 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0773d79385b648d28eb102333c65b748'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:13.895389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1372/2749] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:13 2022] DELETE /identity/v3/users/0773d79385b648d28eb102333c65b748 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:13.900090 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd3fd-a7bf-4cb2-9f0b-4012283ec222 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.900600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd3fd-a7bf-4cb2-9f0b-4012283ec222 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.900700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd3fd-a7bf-4cb2-9f0b-4012283ec222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.919323 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-146cd3fd-a7bf-4cb2-9f0b-4012283ec222 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:13.920581 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-146cd3fd-a7bf-4cb2-9f0b-4012283ec222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:13.993554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1378/2750] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:14.006162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.007941 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.071110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9b438-82d3-4de5-a7dd-88bf1e102941 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.075968 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.076671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.076954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.077224 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.090279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.091903 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.103196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1373/2751] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:14.330217 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.330639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.330958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.355078 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:14.356716 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.438934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1379/2752] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:14.445883 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.447994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.462923 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4365f-e293-42e3-9a69-6304843ebc55 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.479146 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.482124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.482124 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.483209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] PATH_INFO: `/v3/projects/f10b12f0ada647989e12a209aba03f17` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.486060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10b12f0ada647989e12a209aba03f17)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.488603 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.506131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.508229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.517734 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:14.564132 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10b12f0ada647989e12a209aba03f17', 'request_id': 'req-494541f3-35a3-4592-aec2-6672cf9f1d0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.571927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10b12f0ada647989e12a209aba03f17', 'request_id': 'req-494541f3-35a3-4592-aec2-6672cf9f1d0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.580905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10b12f0ada647989e12a209aba03f17', 'request_id': 'req-494541f3-35a3-4592-aec2-6672cf9f1d0b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.592571 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] The token cache is being invalidate because project f10b12f0ada647989e12a209aba03f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:14.592913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.593551 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.595152 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1374/2753] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:14 2022] DELETE /identity/v3/projects/f10b12f0ada647989e12a209aba03f17 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:14.599743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.601265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.603806 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964afed7-dccb-40d6-a32b-81e971a1d451 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.674398 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:14.704607 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.705189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.705475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.705753 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.707813 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.708234 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494541f3-35a3-4592-aec2-6672cf9f1d0b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.714638 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.715803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.716169 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.716509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] PATH_INFO: `/v3/users/d5c70571317a477c9414a3e032bc74b5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.718245 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c70571317a477c9414a3e032bc74b5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.721650 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.780127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.780474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c70571317a477c9414a3e032bc74b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.785058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.790108 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c70571317a477c9414a3e032bc74b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.796713 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1380/2754] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:35:14.804601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c70571317a477c9414a3e032bc74b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.805381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c70571317a477c9414a3e032bc74b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.806173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-217bb945-eb34-4a0f-9ccc-71b06bee34a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c70571317a477c9414a3e032bc74b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:14.824283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1375/2755] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:14 2022] DELETE /identity/v3/users/d5c70571317a477c9414a3e032bc74b5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:15.149762 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.162642 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefeeb2-11e6-4358-816b-ab9483ca379b service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.255417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.256303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.256616 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.256893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] PATH_INFO: `/v3/projects/af2ede11f88a4b0aad71ba636582ba43` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.258224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2ede11f88a4b0aad71ba636582ba43)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.260180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.318671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2ede11f88a4b0aad71ba636582ba43', 'request_id': 'req-198218df-69f1-4ed5-bd7c-370afa94c052'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:15.330062 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2ede11f88a4b0aad71ba636582ba43', 'request_id': 'req-198218df-69f1-4ed5-bd7c-370afa94c052'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:15.337898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2ede11f88a4b0aad71ba636582ba43', 'request_id': 'req-198218df-69f1-4ed5-bd7c-370afa94c052'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:15.361365 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] The token cache is being invalidate because project af2ede11f88a4b0aad71ba636582ba43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:15.361758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:15.362380 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198218df-69f1-4ed5-bd7c-370afa94c052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:15.364014 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1381/2756] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:15 2022] DELETE /identity/v3/projects/af2ede11f88a4b0aad71ba636582ba43 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:15.369742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816e1e-45ca-46e6-9395-8c7ca7d619b3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.370070 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816e1e-45ca-46e6-9395-8c7ca7d619b3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.370395 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816e1e-45ca-46e6-9395-8c7ca7d619b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.392811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-33816e1e-45ca-46e6-9395-8c7ca7d619b3 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:15.394303 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-33816e1e-45ca-46e6-9395-8c7ca7d619b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.417708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.429903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.430320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.430471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.453817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:15.455169 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.465212 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1376/2757] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:15.472048 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.480484 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.480771 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.481109 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.502353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:15.503595 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.513307 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1377/2758] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:15.517849 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.525237 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1382/2759] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:15.528333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab0809-11e0-41c1-abad-0265c2c71854 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.533306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.533981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.534304 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.534546 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.535492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:15.535605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.537442 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.542956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.544797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.555194 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1378/2760] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:15.556363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.567616 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310c672-a68c-471d-85c3-2b02ab1be5f6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.573774 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.575021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.575154 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.575431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.576362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.579220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.611112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1379/2761] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:15.615286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.641076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f2f803-ba86-49d9-9800-271a47db1bf5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.695257 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38b76-6c1a-41a8-b219-554678d217ef admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.700778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.701468 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.701755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.702035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.702744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.704733 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.716261 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.717031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.717348 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.717696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.738726 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1380/2762] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:15.739914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.750411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8305e90-8f47-4866-b99c-37efc2298d94 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.755704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.756329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.756626 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.756905 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.757649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.759306 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.770509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1381/2763] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:15.774566 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.785534 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5c2cf-b1e2-4289-a5db-fc39cf8776f1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.785758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.787312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.793230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.793949 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.794333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.794628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.795390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.797862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a59e99-43bc-48ae-9105-147e415d8e5d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.801697 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1383/2764] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:35:15.809532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1382/2765] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:15.814507 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.825922 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81c9a-4643-4d36-8aa3-ef7e7b230e68 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.831453 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.832485 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.832861 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.833181 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] PATH_INFO: `/v3/projects/8f9221b4f4da48dda7031bc568d5282f/users/36706735ab0d4f76bfd97c0e21e3b8eb/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.837256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.853876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9221b4f4da48dda7031bc568d5282f, user_id=36706735ab0d4f76bfd97c0e21e3b8eb, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.857816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.875050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1384/2766] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:15 2022] PUT /identity/v3/projects/8f9221b4f4da48dda7031bc568d5282f/users/36706735ab0d4f76bfd97c0e21e3b8eb/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:15.887445 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8eb2-b270-4f74-be6c-4aaaa2dc8424 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.888182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8eb2-b270-4f74-be6c-4aaaa2dc8424 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.888579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c8eb2-b270-4f74-be6c-4aaaa2dc8424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.920828 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-394c8eb2-b270-4f74-be6c-4aaaa2dc8424 None None] MFA Rules not processed for user `36706735ab0d4f76bfd97c0e21e3b8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:15.922209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-394c8eb2-b270-4f74-be6c-4aaaa2dc8424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.975678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.006227 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1383/2767] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:16.010755 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.056872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.057538 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.057808 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.076568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:16.077934 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.096024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f1ee7-791f-4323-89ad-dc859b72f43d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.105544 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.106659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.106953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.107227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.107956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.109953 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.139063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1385/2768] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:16.144474 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.154229 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1384/2769] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:16.169200 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.214228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62e558-b2f7-4d00-b0d4-2d7888064650 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.224252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.225160 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.225520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.226174 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.227147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.231212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.250285 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a755c8d-d37c-4fa7-8da3-0ef258f1e533 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.264502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1386/2770] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:16.266841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.282232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aed9fa-38b0-4668-9704-534031b04670 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.287722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.288514 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.288862 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.289250 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.290076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.291898 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.305263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1387/2771] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:16.310260 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.320842 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97226d2f-0d9f-4a2b-81c3-383a083c23b6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.326124 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.327637 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.328039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.328307 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] PATH_INFO: `/v3/projects/b5ea51d18b8f415b99549992c04d7037/users/ba5605db538b44ca8d389809b77b7b0b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.330218 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.330955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.331207 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.331498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.350023 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ea51d18b8f415b99549992c04d7037, user_id=ba5605db538b44ca8d389809b77b7b0b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.352779 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.368517 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.373427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1388/2772] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:16 2022] PUT /identity/v3/projects/b5ea51d18b8f415b99549992c04d7037/users/ba5605db538b44ca8d389809b77b7b0b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:16.379990 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.403881 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.405570 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.414994 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.416793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1385/2773] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:35:16.470995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010979-1f4c-4a2e-84bc-8c5c236634ea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.477567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.478664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.478773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.479084 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.479860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.481770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7d72e-71a6-4cd8-8366-106943d3f60c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.494105 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1389/2774] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:16.500150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.512308 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed6b33-efa9-4791-b7dc-3fae02ed99c1 service swift] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.518690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.519703 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.519986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.520260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] PATH_INFO: `/v3/projects/b5ea51d18b8f415b99549992c04d7037/users/ba5605db538b44ca8d389809b77b7b0b/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.522443 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ea51d18b8f415b99549992c04d7037, user_id=ba5605db538b44ca8d389809b77b7b0b, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.525204 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.541009 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1386/2775] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:16 2022] PUT /identity/v3/projects/b5ea51d18b8f415b99549992c04d7037/users/ba5605db538b44ca8d389809b77b7b0b/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:16.552463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273112f-d159-4d23-a2c5-19b2d85803c0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.552785 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273112f-d159-4d23-a2c5-19b2d85803c0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.553033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273112f-d159-4d23-a2c5-19b2d85803c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.584136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b273112f-d159-4d23-a2c5-19b2d85803c0 None None] MFA Rules not processed for user `ba5605db538b44ca8d389809b77b7b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:16.585532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b273112f-d159-4d23-a2c5-19b2d85803c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.676572 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1390/2776] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:16.688184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.690001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.751336 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79fd3e-5922-467c-a3ca-59e9f1ae9372 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.760955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.764455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.764839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.765162 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.781324 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.782993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.793972 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1387/2777] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:16.799495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.799784 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.800063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.819881 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] MFA Rules not processed for user `5407b0778630459ba6a046d63d69cd02`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:16.821240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.888734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1391/2778] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:16.902939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.982022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55516b-780a-400f-8ec1-812b40e4a9a5 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.987821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47acb0dcc4164628a2442c7dd26d478b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.988608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.989148 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.989485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.001406 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.003370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b004a9-056c-41cf-9e20-6276eef5ae87 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.014006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1388/2779] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:35:18.343794 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.403565 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.417675 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadedac1-fce5-4d31-80ae-f80eba720a2f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.422617 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.423548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.423956 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.424354 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] PATH_INFO: `/v3/users/36706735ab0d4f76bfd97c0e21e3b8eb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.425724 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=36706735ab0d4f76bfd97c0e21e3b8eb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.427688 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.480810 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36706735ab0d4f76bfd97c0e21e3b8eb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:18.481471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36706735ab0d4f76bfd97c0e21e3b8eb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:18.482151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36706735ab0d4f76bfd97c0e21e3b8eb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:18.489310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36706735ab0d4f76bfd97c0e21e3b8eb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:18.498312 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36706735ab0d4f76bfd97c0e21e3b8eb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:18.516234 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1392/2780] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:18 2022] DELETE /identity/v3/users/36706735ab0d4f76bfd97c0e21e3b8eb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:18.520970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.521462 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.521580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.540208 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:18.541422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:18.593841 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.611010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1389/2781] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:18.622192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.623817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:18.685482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66fd4a-3d3e-4c16-9746-5a3d8596c88e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.689513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.690166 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.690450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.690728 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.700350 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.701841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.711490 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1393/2782] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:35:19.008498 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.009986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.023159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d108b1-8f30-4ca3-87d5-fc1555c2475a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.029317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.030219 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.030503 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.030788 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] PATH_INFO: `/v3/projects/8f9221b4f4da48dda7031bc568d5282f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.032624 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9221b4f4da48dda7031bc568d5282f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.034606 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.092854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9221b4f4da48dda7031bc568d5282f', 'request_id': 'req-b72ea071-9433-4700-9067-bcbb9ea838e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.099747 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9221b4f4da48dda7031bc568d5282f', 'request_id': 'req-b72ea071-9433-4700-9067-bcbb9ea838e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.110389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9221b4f4da48dda7031bc568d5282f', 'request_id': 'req-b72ea071-9433-4700-9067-bcbb9ea838e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.122628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] The token cache is being invalidate because project 8f9221b4f4da48dda7031bc568d5282f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:19.122914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.123597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.125301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1390/2783] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:19 2022] DELETE /identity/v3/projects/8f9221b4f4da48dda7031bc568d5282f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:19.129520 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.140555 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3267159-c1c9-4f83-8a3e-4df85662e1ec service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.205391 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.207192 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.207841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.208593 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] PATH_INFO: `/v3/users/ba5605db538b44ca8d389809b77b7b0b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.210996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5605db538b44ca8d389809b77b7b0b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.215244 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.223769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.225286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.234199 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.241532 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ea071-9433-4700-9067-bcbb9ea838e4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.248027 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.249363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.249654 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.250065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] PATH_INFO: `/v3/users/3b100ef05c594d18b3868c2d0cb4f175` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.251848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b100ef05c594d18b3868c2d0cb4f175)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.254318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.284341 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5605db538b44ca8d389809b77b7b0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.285505 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5605db538b44ca8d389809b77b7b0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.286332 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5605db538b44ca8d389809b77b7b0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.295687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5605db538b44ca8d389809b77b7b0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.305570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a9a7ec69-5096-42f6-9597-68e030d28b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5605db538b44ca8d389809b77b7b0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.312960 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b100ef05c594d18b3868c2d0cb4f175'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.320805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b100ef05c594d18b3868c2d0cb4f175'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.325374 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1394/2784] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:19 2022] DELETE /identity/v3/users/ba5605db538b44ca8d389809b77b7b0b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:19.340722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b100ef05c594d18b3868c2d0cb4f175'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.341488 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b100ef05c594d18b3868c2d0cb4f175'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.342130 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b100ef05c594d18b3868c2d0cb4f175'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.357855 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1391/2785] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:19 2022] DELETE /identity/v3/users/3b100ef05c594d18b3868c2d0cb4f175 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:19.362542 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.363031 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.363161 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.382844 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.384195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.452611 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1395/2786] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:19.462913 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.464455 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.523134 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.537088 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9547-cae5-42d1-9565-2a9c4e4abd6d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.540889 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.541553 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.541871 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.542147 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.552195 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.553689 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ac5c4-0887-46cf-8c39-60823919c180 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.565185 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1392/2787] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:19.653940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.655463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.668039 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de2b5-53c7-4f15-8582-6b77b6cebbbb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.673317 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.674382 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.674738 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.675131 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] PATH_INFO: `/v3/projects/b5ea51d18b8f415b99549992c04d7037` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.676473 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ea51d18b8f415b99549992c04d7037)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.678686 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.731765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ea51d18b8f415b99549992c04d7037', 'request_id': 'req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.738667 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ea51d18b8f415b99549992c04d7037', 'request_id': 'req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.745038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ea51d18b8f415b99549992c04d7037', 'request_id': 'req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.759411 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] The token cache is being invalidate because project b5ea51d18b8f415b99549992c04d7037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:19.759707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.760342 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.762073 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1396/2788] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:19 2022] DELETE /identity/v3/projects/b5ea51d18b8f415b99549992c04d7037 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:19.767362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36763-2019-4337-939f-f7cb17506fbb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.767657 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36763-2019-4337-939f-f7cb17506fbb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.770633 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc36763-2019-4337-939f-f7cb17506fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.772289 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.773797 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.788859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bfc36763-2019-4337-939f-f7cb17506fbb None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.790673 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bfc36763-2019-4337-939f-f7cb17506fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.796691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.865651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d692-fa71-4bcd-b8e4-1b48bf7102cf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.870649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.871663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.872028 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.877597 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] PATH_INFO: `/v3/projects/7ef1f8afa02043918b4e76f9af38d47f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.877761 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1393/2789] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:19.878329 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef1f8afa02043918b4e76f9af38d47f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.880680 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.881016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.881310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.881593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.901138 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.902353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.911165 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1394/2790] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:19.915360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.924917 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1088b-e925-4ef6-8b83-2dd875777894 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.929093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef1f8afa02043918b4e76f9af38d47f', 'request_id': 'req-14638186-417c-48ee-8098-e6ffc6ade043'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.935792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef1f8afa02043918b4e76f9af38d47f', 'request_id': 'req-14638186-417c-48ee-8098-e6ffc6ade043'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.942058 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef1f8afa02043918b4e76f9af38d47f', 'request_id': 'req-14638186-417c-48ee-8098-e6ffc6ade043'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.955549 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] The token cache is being invalidate because project 7ef1f8afa02043918b4e76f9af38d47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:19.955835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.956503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14638186-417c-48ee-8098-e6ffc6ade043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:19.958081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1397/2791] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:19 2022] DELETE /identity/v3/projects/7ef1f8afa02043918b4e76f9af38d47f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:19.964080 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93850-4222-4fce-8f52-f2fc8cbcd864 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.964661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93850-4222-4fce-8f52-f2fc8cbcd864 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.965156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e93850-4222-4fce-8f52-f2fc8cbcd864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.985097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.985826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.985931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-77e93850-4222-4fce-8f52-f2fc8cbcd864 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.986142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.986421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.987110 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-77e93850-4222-4fce-8f52-f2fc8cbcd864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.987250 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:19.987472 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.989171 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.995297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1398/2792] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:20.002418 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1395/2793] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:20.007879 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.008082 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.008463 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.008803 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.010010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:20.024002 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e1acc-2d7c-4571-8729-224624a70910 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.028807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:20.029895 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.030047 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:20.030564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.030845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.031120 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.031813 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.033757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.039185 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1399/2794] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:20.043595 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.053754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356648-561a-4bee-9f20-c37d3085d757 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.059950 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.060645 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.060932 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.061209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.061959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.062248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.063013 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1396/2795] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:20.066222 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.067077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.077450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1400/2796] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:20.081814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.091747 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf50d9-6556-4fba-89d6-13c2580d0f11 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.125511 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.137473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58193ac-c468-4cb6-8c86-e8f65ee2c9df admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.143259 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.143953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.144264 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.144568 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.145271 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.147209 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.157059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.157661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.157986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.158252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.159033 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.161184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.171213 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1397/2797] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:20.172293 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.182839 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71fdd8-9cdf-4dd0-9ec6-3267be84eda1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.188044 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.188807 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.189047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.189367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.190075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1401/2798] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:20.190297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.191794 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.194700 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.200071 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.203295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1398/2799] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:20.207112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.217800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7265a5-1ae6-41ab-ae2f-c96da197a446 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.264427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb432de-5840-4399-8b2a-9a2d249a02a8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.269569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.270254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.270540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.270816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.271522 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.273494 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.274609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.275289 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.275563 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.275837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.276594 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.278658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.292381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1399/2800] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:20.295770 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.305708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1402/2801] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:20.307534 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.308220 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b31b7b-f842-4d22-b5cf-671682fafdcd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.315440 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.316611 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.316892 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.317178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] PATH_INFO: `/v3/projects/0d2e4c12bbf94be9a222cf125e5bc02c/users/2f99ad9e740c4464bfa0ad114f444d7a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.318896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db653dbb-9bc2-417d-a7d7-4ec9b81999cd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.324708 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.325406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.325693 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.325969 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.326903 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.329064 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.337854 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2e4c12bbf94be9a222cf125e5bc02c, user_id=2f99ad9e740c4464bfa0ad114f444d7a, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.340154 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1403/2802] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:20.341158 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f955c9f-ef17-47fb-b680-1fedb35ea059 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.344292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.355054 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8b011-6c21-4693-87c4-5e0742e243e5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.358456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1400/2803] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:20 2022] PUT /identity/v3/projects/0d2e4c12bbf94be9a222cf125e5bc02c/users/2f99ad9e740c4464bfa0ad114f444d7a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:20.374153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.374519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.374985 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.408493 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] MFA Rules not processed for user `2f99ad9e740c4464bfa0ad114f444d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:20.410697 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:20.431661 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.432814 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.433097 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.433407 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.434309 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.435958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.446137 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1404/2804] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:20.450746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.461537 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d3f33-b881-4a27-ae41-52912c043f71 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.466641 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.467570 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.467930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.468200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] PATH_INFO: `/v3/projects/bf83328eefb741d086f33cd650fd3942/users/6401f2aaa0af4708953b42863aa25de0/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.483048 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1401/2805] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:20.484457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf83328eefb741d086f33cd650fd3942, user_id=6401f2aaa0af4708953b42863aa25de0, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.487374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.487552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dba17-0eef-4b6c-98bc-340bd86952e6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.500684 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063c854-2164-4cb5-b12c-cc76b5ee585c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.503625 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1405/2806] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:20 2022] PUT /identity/v3/projects/bf83328eefb741d086f33cd650fd3942/users/6401f2aaa0af4708953b42863aa25de0/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:20.508649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.508941 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.509223 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.528704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:20.530009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:20.535509 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.566680 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.567347 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.567631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.567914 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.568692 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.570646 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.582766 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.596624 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1406/2807] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:35:20.605319 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1402/2808] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:20.608545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.609681 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.609934 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:20.670550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152f35a-1c77-47cb-ac77-a8a42b0a53a1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.674793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150b25-7336-435d-9612-62b011a048b2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.679377 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.680060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.680364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.680637 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.681322 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.683691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.709106 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1403/2809] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:20.710179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.720241 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e9918-122f-401c-b8f0-61c890b55df4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.722063 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.722949 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.723207 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.723508 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.725352 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.726017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.726294 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.726569 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.727298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.728935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.733271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.735195 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.742323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1404/2810] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:20.743333 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.745860 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1407/2811] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:20.754268 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cbc6a-2d35-4893-b349-c8fbb3b76732 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.759087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.760006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.760286 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.761079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] PATH_INFO: `/v3/projects/acc2c80f115f4a078099a3c0b7118ea9/users/15502cbc814f45409b71d2550eecde6c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.776861 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc2c80f115f4a078099a3c0b7118ea9, user_id=15502cbc814f45409b71d2550eecde6c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.779597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.793228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1405/2812] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:20 2022] PUT /identity/v3/projects/acc2c80f115f4a078099a3c0b7118ea9/users/15502cbc814f45409b71d2550eecde6c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:20.797198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.807863 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2b8af-aa6f-48e8-ba0e-a73a84e7d84f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.880165 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.881011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.881430 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.881706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.882447 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.884053 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08d555-69bd-4f73-ba28-1820674ad62e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.898846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1408/2813] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:20.900199 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.911255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4bb48-8f7d-4182-9703-cdc2dd441e7d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.916192 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.917116 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.917398 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.917700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] PATH_INFO: `/v3/projects/acc2c80f115f4a078099a3c0b7118ea9/users/15502cbc814f45409b71d2550eecde6c/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.919981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc2c80f115f4a078099a3c0b7118ea9, user_id=15502cbc814f45409b71d2550eecde6c, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.922668 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.934956 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1406/2814] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:20 2022] PUT /identity/v3/projects/acc2c80f115f4a078099a3c0b7118ea9/users/15502cbc814f45409b71d2550eecde6c/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:20.946962 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207404e-c826-4411-b246-da3b378001b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.947254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207404e-c826-4411-b246-da3b378001b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.947532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207404e-c826-4411-b246-da3b378001b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.981823 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4207404e-c826-4411-b246-da3b378001b0 None None] MFA Rules not processed for user `15502cbc814f45409b71d2550eecde6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:20.983138 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4207404e-c826-4411-b246-da3b378001b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:21.027798 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.068848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1409/2815] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:21.081228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.084869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:21.165439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5dff4-cba2-423e-9038-14e27556ed50 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.170278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.170951 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.171236 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.171564 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.182602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.184079 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.194659 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1407/2816] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:21.298706 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207404e-c826-4411-b246-da3b378001b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.309848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207404e-c826-4411-b246-da3b378001b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.314100 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.314765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.315052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.315334 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.381051 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.382623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f816d-e826-4e1b-b4a9-6b89a43c8328 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.394417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1410/2817] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:24.161950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.166409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:24.258164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e8100-257c-4f43-a09e-7aabb5e3ef7a service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.265006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.281438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.281793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.282107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] PATH_INFO: `/v3/users/2f99ad9e740c4464bfa0ad114f444d7a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.283556 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f99ad9e740c4464bfa0ad114f444d7a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.286117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.348210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f99ad9e740c4464bfa0ad114f444d7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:24.358670 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f99ad9e740c4464bfa0ad114f444d7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:24.379269 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f99ad9e740c4464bfa0ad114f444d7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:24.380196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f99ad9e740c4464bfa0ad114f444d7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:24.380970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f99ad9e740c4464bfa0ad114f444d7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:24.408400 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1408/2818] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:24 2022] DELETE /identity/v3/users/2f99ad9e740c4464bfa0ad114f444d7a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:24.413240 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bdf28-dd78-428c-844a-cf1f1cf014d1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.413778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bdf28-dd78-428c-844a-cf1f1cf014d1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.413778 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bdf28-dd78-428c-844a-cf1f1cf014d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.434911 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-786bdf28-dd78-428c-844a-cf1f1cf014d1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:24.436141 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-786bdf28-dd78-428c-844a-cf1f1cf014d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:24.532721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1411/2819] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:24.545314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.547430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:24.628478 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eedd8d-df69-45c6-a32c-ba7d8af9ab0d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.632370 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.633097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.633381 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.633804 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.646394 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.648075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.659986 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1409/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:25.125933 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.126225 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.126503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.139687 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.141255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.161954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0ec4-883b-460c-9e0d-090d109c205e service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.169762 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.170608 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.170907 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.171184 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] PATH_INFO: `/v3/projects/0d2e4c12bbf94be9a222cf125e5bc02c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.172549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2e4c12bbf94be9a222cf125e5bc02c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.174452 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.189024 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] MFA Rules not processed for user `6401f2aaa0af4708953b42863aa25de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:25.191087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.219310 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.262578 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2e4c12bbf94be9a222cf125e5bc02c', 'request_id': 'req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.271021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2e4c12bbf94be9a222cf125e5bc02c', 'request_id': 'req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.282150 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2e4c12bbf94be9a222cf125e5bc02c', 'request_id': 'req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.296060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] The token cache is being invalidate because project 0d2e4c12bbf94be9a222cf125e5bc02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:25.297028 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.297349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.299610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1410/2821] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:25 2022] DELETE /identity/v3/projects/0d2e4c12bbf94be9a222cf125e5bc02c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:25.305396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.307995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.309743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1412/2822] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:25.314136 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.315578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.329035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.335319 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0328d-5494-4525-bf49-601a98c3d6b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.385671 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ee6b2-e3b6-46b2-84fc-43ff5e0cbca2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.391632 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.392729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.393586 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.393958 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] PATH_INFO: `/v3/users/15502cbc814f45409b71d2550eecde6c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.395334 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=15502cbc814f45409b71d2550eecde6c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.397685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.413830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.414306 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.414532 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.415003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.416495 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.418913 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.448021 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1413/2823] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:25 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:25.452957 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.454896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.458805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15502cbc814f45409b71d2550eecde6c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.466522 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15502cbc814f45409b71d2550eecde6c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.483011 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15502cbc814f45409b71d2550eecde6c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.483919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15502cbc814f45409b71d2550eecde6c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.485256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15502cbc814f45409b71d2550eecde6c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:25.514210 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1411/2824] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:25 2022] DELETE /identity/v3/users/15502cbc814f45409b71d2550eecde6c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:25.539469 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f03545-20ec-4ff5-b30a-6a22fbcb81cb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.615212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.616032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.616406 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.616754 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.617890 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.620400 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.647201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1414/2825] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:25 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:25.651704 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.654010 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.674298 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5748f2f-51be-4adc-9bdf-c45c7958a82f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.679519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.680389 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.680844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.682504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.682504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.684732 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.702757 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1412/2826] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:25.707050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.718471 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68370967-c1f5-4df9-aecd-998bf3629720 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.723315 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.724221 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.727809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.727809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] PATH_INFO: `/v3/projects/055f40a719824e82b388537fd163f597/users/f196d193c7f24e60a2935edc6727aa54/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.751018 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055f40a719824e82b388537fd163f597, user_id=f196d193c7f24e60a2935edc6727aa54, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.753792 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.772905 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1415/2827] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:25 2022] PUT /identity/v3/projects/055f40a719824e82b388537fd163f597/users/f196d193c7f24e60a2935edc6727aa54/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:25.776837 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.790035 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378ed46-e395-4b1d-bb55-785e6f0ecafe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.804862 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.878388 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.878874 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.879107 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.879385 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.880634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.881999 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.896367 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1413/2828] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:25.897986 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.905042 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.913920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe56f5-4730-4628-a0f3-8e12fc5d9619 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.919729 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.921569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.922060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.922535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] PATH_INFO: `/v3/projects/055f40a719824e82b388537fd163f597/users/f196d193c7f24e60a2935edc6727aa54/roles/abfc6b6d02e0407bbaef10a05d71ba62` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.928995 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.929170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055f40a719824e82b388537fd163f597, user_id=f196d193c7f24e60a2935edc6727aa54, role_id=abfc6b6d02e0407bbaef10a05d71ba62)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.930783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.932427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807723b-e945-4b7b-99a6-fba05ac8b793 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.945301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78954c-46e7-42e0-8143-dc68148cb86b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.952719 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1416/2829] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:25 2022] PUT /identity/v3/projects/055f40a719824e82b388537fd163f597/users/f196d193c7f24e60a2935edc6727aa54/roles/abfc6b6d02e0407bbaef10a05d71ba62 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:26.024778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.025682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.025966 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.026433 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] PATH_INFO: `/v3/projects/acc2c80f115f4a078099a3c0b7118ea9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.028904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc2c80f115f4a078099a3c0b7118ea9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.030353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.113786 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc2c80f115f4a078099a3c0b7118ea9', 'request_id': 'req-63a68521-9133-4b6a-924c-ae866adfe8ad'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:26.124242 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc2c80f115f4a078099a3c0b7118ea9', 'request_id': 'req-63a68521-9133-4b6a-924c-ae866adfe8ad'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:26.138275 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc2c80f115f4a078099a3c0b7118ea9', 'request_id': 'req-63a68521-9133-4b6a-924c-ae866adfe8ad'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:26.154327 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] The token cache is being invalidate because project acc2c80f115f4a078099a3c0b7118ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:26.154631 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:26.155592 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63a68521-9133-4b6a-924c-ae866adfe8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:26.157890 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1414/2830] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:25 2022] DELETE /identity/v3/projects/acc2c80f115f4a078099a3c0b7118ea9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:26.162959 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.163287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.163563 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.192920 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:26.194259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:26.273677 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1417/2831] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:26.275939 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.276473 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.276580 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.314658 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:26.316003 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:26.327609 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1415/2832] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:26.332259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.350189 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c8d19-0655-4b54-bb67-f92d430ba55f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.360059 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.360846 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.361229 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.362246 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.362740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:26.363147 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.364981 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.380429 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1418/2833] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:26.384729 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.395356 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df5f1-68dd-40d5-b0af-30bec68b6a2f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.401975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.402739 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.403023 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.403346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.405007 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.408170 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.451246 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1416/2834] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:26.455498 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.471446 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59f14b-1518-4204-827a-ddcadba4cc51 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.540208 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.541156 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.541440 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.541746 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.542464 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.548297 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.572231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1419/2835] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:26.573431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.585128 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16acc2-2bd6-4c45-8b5a-c4d53e25424e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.596605 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.597479 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.597775 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.598091 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.599153 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.601060 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.617798 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1417/2836] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:26.619267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.631216 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bf505-e145-48e7-9f7d-76b053e1e1f2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.637977 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.639303 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.639610 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.640022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.640872 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.642531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652f1be-fc02-4845-bbf5-6bc9030f6d67 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.658935 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1420/2837] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:26.660755 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.666374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.672996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403c3d4-688c-4667-9671-208420ec70f9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.679297 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.680707 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.680990 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.681261 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] PATH_INFO: `/v3/projects/e3629a43f3544c79adba86ba6ed7cd85/users/8a399f38aefe45cdae3734d5f1ee12e8/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.712110 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3629a43f3544c79adba86ba6ed7cd85, user_id=8a399f38aefe45cdae3734d5f1ee12e8, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.715188 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.732095 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1418/2838] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:26 2022] PUT /identity/v3/projects/e3629a43f3544c79adba86ba6ed7cd85/users/8a399f38aefe45cdae3734d5f1ee12e8/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:26.743618 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.743947 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.744224 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.784050 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] MFA Rules not processed for user `8a399f38aefe45cdae3734d5f1ee12e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:26.786197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:26.853808 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.890857 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1421/2839] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:26.903800 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.906619 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:26.983075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e0780-28e6-4021-94fe-4739484e3132 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.988894 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.989573 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.989859 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.990143 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.000161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.001797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e46ea7-8640-4d54-80cf-3ea246251d39 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.012700 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1419/2840] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:27.500761 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.502276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:27.643390 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50fb7-50a2-48ef-b4e1-debcb4a5a4af None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.649961 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.651087 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.651678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.652238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] PATH_INFO: `/v3/users/defc63ed7ff84a40af71a475db376580` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.654756 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=defc63ed7ff84a40af71a475db376580)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.661060 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.754587 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defc63ed7ff84a40af71a475db376580'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:27.755366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defc63ed7ff84a40af71a475db376580'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:27.756267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defc63ed7ff84a40af71a475db376580'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:27.766252 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defc63ed7ff84a40af71a475db376580'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:27.778200 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defc63ed7ff84a40af71a475db376580'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:27.800696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1422/2841] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:27 2022] DELETE /identity/v3/users/defc63ed7ff84a40af71a475db376580 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:27.806316 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.807634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.808235 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.832579 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:27.834103 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:27.938309 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1420/2842] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:27.947337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.949873 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:28.023487 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2364c4-ef7c-44fa-81cc-4eec43d36718 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.029576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.031126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.031482 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.031996 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.047043 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.048669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3244-5712-4d8f-bfac-13f7ff5baee5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.063226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1423/2843] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:29.243260 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.244863 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:29.288518 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffd147-91ec-45a9-b1d0-c2dc7b7df823 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.304978 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.306519 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.306816 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.307470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] PATH_INFO: `/v3/projects/62db9432d77f4e24abb5f506fd8b7f4e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.309545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62db9432d77f4e24abb5f506fd8b7f4e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.315329 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.448829 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62db9432d77f4e24abb5f506fd8b7f4e', 'request_id': 'req-42080d4c-e917-4e39-bbd1-6b07e061d4d4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:29.472904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62db9432d77f4e24abb5f506fd8b7f4e', 'request_id': 'req-42080d4c-e917-4e39-bbd1-6b07e061d4d4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:29.495709 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62db9432d77f4e24abb5f506fd8b7f4e', 'request_id': 'req-42080d4c-e917-4e39-bbd1-6b07e061d4d4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:29.527082 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] The token cache is being invalidate because project 62db9432d77f4e24abb5f506fd8b7f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:29.527664 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:29.528525 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:29.530386 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1421/2844] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:29 2022] DELETE /identity/v3/projects/62db9432d77f4e24abb5f506fd8b7f4e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:31.571075 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.571621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.571621 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.586731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:31.611758 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] MFA Rules not processed for user `f196d193c7f24e60a2935edc6727aa54`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:31.612343 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:31.749909 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1424/2845] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:31.758840 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:31.761045 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:31.768768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:31.889016 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42080d4c-e917-4e39-bbd1-6b07e061d4d4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:31.895496 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:31.896825 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.897430 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.897965 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.901810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:32.021778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.023745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb720fc-37ba-43a9-8916-7f529a001c8d service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:32.034708 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1422/2846] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:41.738101 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=31 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:42.029715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=54 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.340353 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.341945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:45.413163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdfbd6-29d0-4a20-b52e-9d3c13a6cd8f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.418367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.419255 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.419540 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.419815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] PATH_INFO: `/v3/users/8a399f38aefe45cdae3734d5f1ee12e8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.421543 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a399f38aefe45cdae3734d5f1ee12e8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.424580 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.485608 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a399f38aefe45cdae3734d5f1ee12e8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:45.486780 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a399f38aefe45cdae3734d5f1ee12e8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:45.487815 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a399f38aefe45cdae3734d5f1ee12e8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:45.498115 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a399f38aefe45cdae3734d5f1ee12e8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:45.509384 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a399f38aefe45cdae3734d5f1ee12e8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:45.527151 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1425/2847] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:35:45 2022] DELETE /identity/v3/users/8a399f38aefe45cdae3734d5f1ee12e8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:45.532175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.532742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.532742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.554690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:45.556065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:45.627629 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1423/2848] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:45.641828 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.643271 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:45.706233 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34f68-95ee-4e74-901a-d5e154753f36 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.711930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.712833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.713254 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.713576 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.727659 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.729106 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a75818-694f-4f4c-b2fb-37610e3d0f53 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.740672 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1426/2849] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:35:46.043318 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.045069 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.064926 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2e14c-30d7-441b-882d-19293386f6e0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.070046 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.072809 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.073098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.073425 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] PATH_INFO: `/v3/projects/e3629a43f3544c79adba86ba6ed7cd85` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.074683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3629a43f3544c79adba86ba6ed7cd85)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.076730 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.127878 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3629a43f3544c79adba86ba6ed7cd85', 'request_id': 'req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:46.134832 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3629a43f3544c79adba86ba6ed7cd85', 'request_id': 'req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:46.143436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3629a43f3544c79adba86ba6ed7cd85', 'request_id': 'req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:46.155738 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] The token cache is being invalidate because project e3629a43f3544c79adba86ba6ed7cd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:35:46.156032 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:46.156810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e2acc479-7c35-4b97-b26c-bdd6ea9e864c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:35:46.158477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1424/2850] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:35:46 2022] DELETE /identity/v3/projects/e3629a43f3544c79adba86ba6ed7cd85 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:46.163602 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.163999 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.164418 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.186197 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:46.187431 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.207180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.261928 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1427/2851] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:46.271253 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.271547 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.271826 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.292173 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:46.293421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.302490 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1425/2852] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:46.307633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.317892 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e4a1c-51ed-452e-be14-a76503ebf8e1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.324273 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.324970 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.325253 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.325531 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.326267 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:46.326612 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.328362 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.341367 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1428/2853] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:46.346058 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.355973 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec402b-3c6f-4018-98da-d112405698ec None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.361495 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.362237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.362521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.362797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.363476 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.365431 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.396353 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1426/2854] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:46.400833 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.411664 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850f58d-47dc-4d33-8df0-7295df292ab6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.471983 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.472682 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.472963 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.473232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.473914 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.475848 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.496176 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1429/2855] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:46.500504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.511193 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7af8-48fc-4f8b-92a6-0477b85a836b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.516702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.517380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.517690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.517964 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.518701 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.520358 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.532481 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1427/2856] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:46.536807 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.549129 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af57081-4643-42b2-950a-baca20045e18 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.554404 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.555180 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.555457 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.555740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.556528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.559188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8baaa45-9c15-46b1-b004-baef00bbf465 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.572651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1430/2857] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:46.577022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.589047 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08cde5c-0ea4-41ae-bdc9-f5e01c5c42fb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.594618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.595523 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.595803 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.596098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] PATH_INFO: `/v3/projects/a262104aa20b4123a7f2651f7e4a6ece/users/975602ffcbaa4457a3b160a83185542a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.618287 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a262104aa20b4123a7f2651f7e4a6ece, user_id=975602ffcbaa4457a3b160a83185542a, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.621022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.637017 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1428/2858] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:35:46 2022] PUT /identity/v3/projects/a262104aa20b4123a7f2651f7e4a6ece/users/975602ffcbaa4457a3b160a83185542a/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:46.647958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10ed9-e62f-4622-abce-2e83ef4952be None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.648248 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10ed9-e62f-4622-abce-2e83ef4952be None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.648554 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10ed9-e62f-4622-abce-2e83ef4952be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.677188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cfd10ed9-e62f-4622-abce-2e83ef4952be None None] MFA Rules not processed for user `975602ffcbaa4457a3b160a83185542a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:46.678428 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cfd10ed9-e62f-4622-abce-2e83ef4952be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.749541 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1431/2859] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:46.761744 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.763835 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.788843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.829374 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86300c-7a44-4638-924e-3b8a0add73b0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.833317 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '62ae50de406e46dbb2448a8f04426bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.833975 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.834255 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.834559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.845411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.847767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.862256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1429/2860] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:35:50.570545 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.571910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.571910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.594850 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] MFA Rules not processed for user `6401f2aaa0af4708953b42863aa25de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:50.595940 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:50.611198 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.663052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1432/2861] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:35:56.855254 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=17 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:00.656126 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=17 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:11.997438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:11.999411 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:12.070226 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c181ea-5ac8-4576-9241-eaa0902fc0f0 service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:12.074340 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:12.074996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:12.075273 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:12.075540 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:12.077649 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:12.142810 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:12.144278 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da90d0-6a06-4838-a48e-bb81fe640d43 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:12.155283 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1430/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:12.721011 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:12.791175 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adf7cd-6ed0-441c-82f2-97d2046e6c27 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:12.795744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:12.796767 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:12.797052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:12.797357 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] PATH_INFO: `/v3/users/975602ffcbaa4457a3b160a83185542a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:12.798578 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] RBAC: Authorizing `identity:delete_user(user_id=975602ffcbaa4457a3b160a83185542a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:12.800553 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:12.852291 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975602ffcbaa4457a3b160a83185542a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:12.853127 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975602ffcbaa4457a3b160a83185542a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:12.853826 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975602ffcbaa4457a3b160a83185542a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:12.861752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975602ffcbaa4457a3b160a83185542a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:12.875655 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975602ffcbaa4457a3b160a83185542a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:12.898439 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1433/2863] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:36:12 2022] DELETE /identity/v3/users/975602ffcbaa4457a3b160a83185542a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:12.902940 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:12.903793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:12.903793 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:12.921982 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:12.923239 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:12.988270 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1431/2864] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:36:13.000649 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.002272 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:13.074930 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235895c8-d19f-4810-be22-9eb1c4cc947f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.078567 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.079232 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.079513 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.079789 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.089209 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.090678 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e9c1-577a-4e75-b044-61ba4e2f7bee service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.100841 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1434/2865] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:13.401970 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.403475 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:13.420470 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b5474-7369-4c03-a756-df3b156b2911 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.425505 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.426409 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.426685 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.426954 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] PATH_INFO: `/v3/projects/a262104aa20b4123a7f2651f7e4a6ece` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.428618 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a262104aa20b4123a7f2651f7e4a6ece)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.430601 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.499919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:13.503994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a262104aa20b4123a7f2651f7e4a6ece', 'request_id': 'req-81672313-7f30-489c-807a-b406812a8474'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:13.512996 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a262104aa20b4123a7f2651f7e4a6ece', 'request_id': 'req-81672313-7f30-489c-807a-b406812a8474'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:13.522427 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a262104aa20b4123a7f2651f7e4a6ece', 'request_id': 'req-81672313-7f30-489c-807a-b406812a8474'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:13.535585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] The token cache is being invalidate because project a262104aa20b4123a7f2651f7e4a6ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:36:13.535902 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:13.536593 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-81672313-7f30-489c-807a-b406812a8474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:13.538331 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1432/2866] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:36:13 2022] DELETE /identity/v3/projects/a262104aa20b4123a7f2651f7e4a6ece => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:13.543281 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.543575 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.543855 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.562435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:13.563665 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:13.581503 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:13.629231 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1435/2867] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:36:13.638607 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.638919 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.639201 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.658715 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:13.659971 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:13.668690 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1433/2868] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:36:13.673035 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.684310 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace18736-9d55-4087-ab56-8c02892ca1c1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.689887 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.690740 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.691019 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.691287 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.692036 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:13.692359 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.694468 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.707816 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1436/2869] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jun 27 15:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:13.712806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.725933 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636766d8-000c-45f2-aecc-f2fae0afe55d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.731136 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.731742 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.732026 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.732301 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.733029 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.734950 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.765676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1434/2870] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:13.770634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.791150 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62aa05-2860-4a58-8064-54a99c762804 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.875987 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.876707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.876992 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.877268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.877966 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.879989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.903691 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1437/2871] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:13.908653 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.920145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfe63a-295a-42df-afb0-adcca5c53c22 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.926330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.926663 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.926952 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.927230 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.928001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.929691 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.942097 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1435/2872] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:36:13.948339 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.959052 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02c696-8f6b-4161-82e3-ed208b53efe3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.964299 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.965268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.965268 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.965600 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.966294 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.967919 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09bd07-73f6-47c0-86f7-771cf4583ee2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.980228 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1438/2873] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jun 27 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:36:13.984190 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.995155 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757747b-8b1a-4e32-9852-3ba39d148bd9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:14.000022 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:14.002590 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.002893 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.003175 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] PATH_INFO: `/v3/projects/8d716ab5de974e75a73201a0d4a37a20/users/62d52ddcea4b4950b4d01cdd3c5ec307/roles/0a2c65ac0f7c41fd8d3b88920a4f73de` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.021676 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d716ab5de974e75a73201a0d4a37a20, user_id=62d52ddcea4b4950b4d01cdd3c5ec307, role_id=0a2c65ac0f7c41fd8d3b88920a4f73de)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:14.024474 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:14.043521 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1436/2874] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jun 27 15:36:13 2022] PUT /identity/v3/projects/8d716ab5de974e75a73201a0d4a37a20/users/62d52ddcea4b4950b4d01cdd3c5ec307/roles/0a2c65ac0f7c41fd8d3b88920a4f73de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:14.123958 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.124300 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.124592 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.159515 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] MFA Rules not processed for user `62d52ddcea4b4950b4d01cdd3c5ec307`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:14.160827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:14.245022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1439/2875] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:36:14.257904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:14.268172 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:14.338504 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf247d-67bc-456b-b86e-063f2441427a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:14.345702 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b9a1a77fd5b40f6b531c79d69044430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:14.346463 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.346811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.347161 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.358831 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:14.360392 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a38d0-dcf5-45c9-90a4-d74bccb4c760 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:14.372424 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1437/2876] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:14.398551 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:14.421201 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:14.477296 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65ada-65a0-4168-93dc-e0e05b65eea9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:14.481829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:14.482895 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.483117 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.483344 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] PATH_INFO: `/v3/users/62d52ddcea4b4950b4d01cdd3c5ec307` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.484614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d52ddcea4b4950b4d01cdd3c5ec307)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:14.486594 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:14.540017 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d52ddcea4b4950b4d01cdd3c5ec307'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:14.540696 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d52ddcea4b4950b4d01cdd3c5ec307'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:14.541288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d52ddcea4b4950b4d01cdd3c5ec307'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:14.548671 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d52ddcea4b4950b4d01cdd3c5ec307'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:14.557769 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d52ddcea4b4950b4d01cdd3c5ec307'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:14.576417 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1440/2877] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:36:14 2022] DELETE /identity/v3/users/62d52ddcea4b4950b4d01cdd3c5ec307 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:14.582117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.582628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.582741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.601295 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] MFA Rules not processed for user `d7a4d53e682240199113e94049a9ab62`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:14.602830 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:14.668428 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1438/2878] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:36:14.678277 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:14.679730 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:14.739904 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e100d-3e35-43d4-97ef-5ba084734026 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:14.747196 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:14.747869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:14.748274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:14.748550 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:14.759731 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:14.761298 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:14.772569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1441/2879] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:15.062927 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:15.064549 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:15.079145 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466552f9-5f68-4d47-8343-26a420abcef2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:15.085438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:15.091302 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:15.091591 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:15.091869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] PATH_INFO: `/v3/projects/8d716ab5de974e75a73201a0d4a37a20` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:15.093314 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d716ab5de974e75a73201a0d4a37a20)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:15.095346 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:15.104359 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:15.147112 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d716ab5de974e75a73201a0d4a37a20', 'request_id': 'req-85368de2-ff39-4cee-ae88-3dcbc69720e0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:15.153223 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d716ab5de974e75a73201a0d4a37a20', 'request_id': 'req-85368de2-ff39-4cee-ae88-3dcbc69720e0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:15.162542 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d716ab5de974e75a73201a0d4a37a20', 'request_id': 'req-85368de2-ff39-4cee-ae88-3dcbc69720e0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:15.174196 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] The token cache is being invalidate because project 8d716ab5de974e75a73201a0d4a37a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:36:15.174485 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:15.175189 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:36:15.176931 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1439/2880] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:36:15 2022] DELETE /identity/v3/projects/8d716ab5de974e75a73201a0d4a37a20 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:17.702927 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:17.704946 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:17.724427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef0a80-8292-46f3-9249-fd454c976303 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:17.765046 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:17.780118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:17.781098 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:17.781528 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:17.781900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:17.784765 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:17.867989 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:17.869435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:17.882752 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1442/2881] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:19.069021 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:19.070662 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:19.085159 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85368de2-ff39-4cee-ae88-3dcbc69720e0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:19.089613 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:19.090272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:19.090562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:19.090844 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:19.092969 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:19.107396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:19.108904 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:19.120872 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1440/2882] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:27.876497 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=36 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:29.115501 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:31.624734 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:31.634960 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906e62e-2905-4420-b6c7-542b2e976714 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:31.639791 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:31.640472 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:31.640975 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:31.641263 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:31.643735 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:31.654965 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:31.656476 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:31.666193 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1443/2883] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:31.782602 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:31.791537 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461f38f-48c2-497f-8b5c-0d1c61d178b0 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:31.795745 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:31.796432 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:31.796718 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:31.796993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:31.798853 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:31.813752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:31.815197 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:31.827300 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1441/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:36:41.662337 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:41.821610 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:31.261695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:31.263410 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:31.277435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c015625-31ad-4843-8ff7-bd4267e117c6 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:31.283381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:31.284032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:31.284304 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:31.284633 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:31.286820 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:31.297835 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:31.299869 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:31.308475 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1444/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:37:41.304465 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:55.655597 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:55.657935 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:55.675343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f94783-1bd5-42ef-a665-fa805108245d service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:55.681466 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:55.683087 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:55.683585 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:55.684055 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:55.685944 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:55.699393 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:55.701053 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:55.712814 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1442/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:37:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:37:57.758239 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:57.771827 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669c589-ff89-4a8c-9514-ea0f8b47094a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:57.778226 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:57.778865 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:57.779164 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:57.779435 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:57.783034 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:57.796552 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:57.798152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:57.810276 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1445/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:38:04.455768 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:04.467615 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cacae-5edc-4cea-962f-155c3964f9f6 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:04.472811 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:04.473526 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:04.473806 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:04.474073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:04.476182 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:04.550749 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:04.552422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:04.564117 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1443/2888] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:38:06.202292 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:06.204654 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:06.219876 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce981c-1f96-4a7c-960e-504b4dd66f05 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:06.227921 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:06.229152 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:06.229427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:06.229786 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] PATH_INFO: `/v3/users/b35218bf5d284771b5107601ec633f7c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:06.231569 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35218bf5d284771b5107601ec633f7c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:06.234639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:06.288918 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35218bf5d284771b5107601ec633f7c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.290184 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35218bf5d284771b5107601ec633f7c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.291016 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35218bf5d284771b5107601ec633f7c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.298830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35218bf5d284771b5107601ec633f7c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.308191 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35218bf5d284771b5107601ec633f7c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.327140 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1446/2889] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:38:06 2022] DELETE /identity/v3/users/b35218bf5d284771b5107601ec633f7c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:38:06.518279 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:06.529769 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada3a8b-ae00-4595-94b1-310cd9cc8641 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:06.592679 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:06.593559 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:06.593845 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:06.594105 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] PATH_INFO: `/v3/projects/9954040eb98d4ecdafbc681132689ff5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:06.595343 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9954040eb98d4ecdafbc681132689ff5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:06.597325 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:06.647237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9954040eb98d4ecdafbc681132689ff5', 'request_id': 'req-3ef824d1-364f-485e-bc71-59495b536872'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.653873 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9954040eb98d4ecdafbc681132689ff5', 'request_id': 'req-3ef824d1-364f-485e-bc71-59495b536872'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.662962 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9954040eb98d4ecdafbc681132689ff5', 'request_id': 'req-3ef824d1-364f-485e-bc71-59495b536872'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.674064 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] The token cache is being invalidate because project 9954040eb98d4ecdafbc681132689ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:38:06.674354 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.674981 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.676562 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1444/2890] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:38:06 2022] DELETE /identity/v3/projects/9954040eb98d4ecdafbc681132689ff5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:38:06.681749 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:06.683502 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:06.697809 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83744b7f-b385-4337-be2d-1a4aa17b4ac0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:06.758003 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:06.758893 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:06.759170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:06.759478 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] PATH_INFO: `/v3/users/ca5afb0b572d4ca3b6f7094ada8bc622` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:06.761081 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5afb0b572d4ca3b6f7094ada8bc622)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:06.763029 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:06.811320 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5afb0b572d4ca3b6f7094ada8bc622'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.811900 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5afb0b572d4ca3b6f7094ada8bc622'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.812535 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5afb0b572d4ca3b6f7094ada8bc622'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.820256 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5afb0b572d4ca3b6f7094ada8bc622'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.824687 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:06.829875 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f58a9a1-fa74-4101-8e9c-e18165d59361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5afb0b572d4ca3b6f7094ada8bc622'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:06.850988 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1447/2891] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:38:06 2022] DELETE /identity/v3/users/ca5afb0b572d4ca3b6f7094ada8bc622 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:38:07.031805 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:07.033323 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:07.047390 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef824d1-364f-485e-bc71-59495b536872 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:07.057123 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:07.119228 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:07.120042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:07.120349 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:07.120634 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] PATH_INFO: `/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:07.121936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d3dc0907f54b7d9896aa1eef5d0df4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:07.124581 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:07.180371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d3dc0907f54b7d9896aa1eef5d0df4', 'request_id': 'req-ddadbe5d-ee69-4c22-a5a5-9519ef789256'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:07.190075 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d3dc0907f54b7d9896aa1eef5d0df4', 'request_id': 'req-ddadbe5d-ee69-4c22-a5a5-9519ef789256'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:07.204778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d3dc0907f54b7d9896aa1eef5d0df4', 'request_id': 'req-ddadbe5d-ee69-4c22-a5a5-9519ef789256'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:07.216362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] The token cache is being invalidate because project 11d3dc0907f54b7d9896aa1eef5d0df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:38:07.216656 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:07.217380 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:38:07.220135 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1445/2892] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:38:07 2022] DELETE /identity/v3/projects/11d3dc0907f54b7d9896aa1eef5d0df4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:38:14.959695 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:14.960259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:14.960259 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:14.981839 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] MFA Rules not processed for user `6401f2aaa0af4708953b42863aa25de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:14.983274 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:15.046009 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1448/2893] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:38:15.388724 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.390330 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:15.451407 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadbe5d-ee69-4c22-a5a5-9519ef789256 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:15.455752 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:15.456438 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:15.456722 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:15.456993 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:15.458848 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:15.468869 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:15.470291 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.480042 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1446/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:38:16.975978 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:16.977623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:17.057491 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2df444-3a23-43ec-aa16-a976cea9f8a7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:17.064489 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:17.065614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:17.065614 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:17.065886 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:17.067745 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:17.079538 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:17.081227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:17.091634 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1449/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:38:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:38:25.475682 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:27.086995 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=65,keystone:DELETE=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:30.321006 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:30.322776 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:30.404543 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e92076-86d2-4af3-8214-e2ab6ba01ab1 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:30.411683 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3f4d2d7923a4a9a8f59968f770949d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:30.412372 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:30.412815 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:30.413083 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:30.414953 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:30.428946 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:30.430550 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763ab02-d363-4de7-8217-be03e86a43a8 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:30.441955 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1447/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:38:40.437001 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=34 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:32.680188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:32.681864 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:32.773283 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3662a-dab5-44c1-898e-4db0f138d6d8 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:32.779462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:32.780118 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:32.780427 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:32.780707 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:32.782725 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:32.794743 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:32.796364 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:32.806836 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1450/2897] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:39:42.802669 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=37 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:09.061767 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:09.062344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:09.062344 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:09.080947 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] MFA Rules not processed for user `6401f2aaa0af4708953b42863aa25de0`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:09.082178 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:09.090311 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1448/2898] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jun 27 15:40:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:09.450639 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:09.452483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:09.492795 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6328-7cd8-43a6-b1e9-191331b5b9b1 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:09.497676 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:09.498360 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:09.498647 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:09.498910 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:09.501032 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:09.512366 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:09.513829 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:09.522824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1451/2899] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:19.075797 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:19.519235 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=15 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:55.498360 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:55.510073 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb39ea-ce18-44f1-bf36-cb7dd9981646 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:55.514174 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:55.514821 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:55.515098 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:55.515348 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:55.517678 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:55.530778 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:55.533238 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:55.543843 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1449/2900] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:56.633773 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:56.642967 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38969fa9-0db2-42c7-ab1a-b16cfff76570 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:56.649076 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:56.649997 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:56.650770 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:56.651238 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:56.653721 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:56.687437 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:56.689227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:56.709651 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1452/2901] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:57.902422 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:57.913834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67255baf-b92b-4314-a2fa-24279a7a3b83 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:57.920288 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:57.921237 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:57.921576 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:57.921994 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:57.925000 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:57.939454 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:57.940943 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:57.951936 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1450/2902] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:06.707022 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=18 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:07.947363 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=12 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:08.351481 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:08.360395 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fae8d-8f4f-4b58-8f87-367ee31850fb service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:08.364227 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16969e4969c4457eaec3ed37c5757aa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:08.364907 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:08.365188 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:08.365455 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:08.367345 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:08.454445 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:08.455748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:08.474843 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1453/2903] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:11.446164 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:11.458439 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713a705-b930-4225-93ba-2c3c43c404ba service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:11.466065 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:11.467492 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:11.467984 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:11.468524 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:11.471780 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:11.486456 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:11.488272 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:11.517421 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1451/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:42:14.745772 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:14.760704 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381cdb5-ea92-4852-ad13-fbeb31058902 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:14.766623 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:14.767462 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:14.767817 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:14.768163 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:14.770477 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:14.785130 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:14.787288 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:14.807364 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:14.808909 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:14.813931 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1454/2905] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:14.836096 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7f4ac-ad42-4de8-b24a-6762fb3c642c service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:14.848142 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:14.849180 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:14.849531 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:14.849819 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] PATH_INFO: `/v3/users/6401f2aaa0af4708953b42863aa25de0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:14.851076 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6401f2aaa0af4708953b42863aa25de0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:14.853068 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:14.902743 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6401f2aaa0af4708953b42863aa25de0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:14.909834 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6401f2aaa0af4708953b42863aa25de0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:14.921783 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6401f2aaa0af4708953b42863aa25de0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:14.922444 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6401f2aaa0af4708953b42863aa25de0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:14.923396 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6401f2aaa0af4708953b42863aa25de0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:14.937741 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1452/2906] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:42:14 2022] DELETE /identity/v3/users/6401f2aaa0af4708953b42863aa25de0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:42:15.132215 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:15.145640 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f83ca-fa3f-466d-ad5e-b7198be2ed9c service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:15.213744 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:15.214539 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:15.214824 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:15.215093 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] PATH_INFO: `/v3/projects/bf83328eefb741d086f33cd650fd3942` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:15.216483 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf83328eefb741d086f33cd650fd3942)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:15.218450 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:15.273830 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf83328eefb741d086f33cd650fd3942', 'request_id': 'req-92eff435-5bea-41ce-95cf-524fb89c2a55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.277548 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:15.279781 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf83328eefb741d086f33cd650fd3942', 'request_id': 'req-92eff435-5bea-41ce-95cf-524fb89c2a55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.287159 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf83328eefb741d086f33cd650fd3942', 'request_id': 'req-92eff435-5bea-41ce-95cf-524fb89c2a55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.301748 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] The token cache is being invalidate because project bf83328eefb741d086f33cd650fd3942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:42:15.302038 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.302663 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.304262 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1455/2907] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:42:15 2022] DELETE /identity/v3/projects/bf83328eefb741d086f33cd650fd3942 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:42:15.308533 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:15.310066 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:15.323817 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ce96-7bf4-4d54-8534-45e7073576ea admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:15.387077 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:15.387979 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:15.388265 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:15.388628 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] PATH_INFO: `/v3/users/f196d193c7f24e60a2935edc6727aa54` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:15.389827 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f196d193c7f24e60a2935edc6727aa54)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:15.391795 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:15.448037 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f196d193c7f24e60a2935edc6727aa54'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.455256 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f196d193c7f24e60a2935edc6727aa54'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.459758 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:15.469487 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f196d193c7f24e60a2935edc6727aa54'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.470362 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f196d193c7f24e60a2935edc6727aa54'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.471262 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f196d193c7f24e60a2935edc6727aa54'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.488545 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1453/2908] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jun 27 15:42:15 2022] DELETE /identity/v3/users/f196d193c7f24e60a2935edc6727aa54 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:42:15.671945 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:15.674212 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:15.695896 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff435-5bea-41ce-95cf-524fb89c2a55 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:15.767386 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a4d53e682240199113e94049a9ab62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9702a09cdfa2459d8590360a884e14e2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:15.768556 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:15.768840 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:15.769135 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] PATH_INFO: `/v3/projects/055f40a719824e82b388537fd163f597` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:15.770376 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055f40a719824e82b388537fd163f597)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:15.773123 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:15.823170 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055f40a719824e82b388537fd163f597', 'request_id': 'req-1b357416-091d-419b-90d4-66fe278262a5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.828722 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055f40a719824e82b388537fd163f597', 'request_id': 'req-1b357416-091d-419b-90d4-66fe278262a5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.835980 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055f40a719824e82b388537fd163f597', 'request_id': 'req-1b357416-091d-419b-90d4-66fe278262a5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.849629 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] The token cache is being invalidate because project 055f40a719824e82b388537fd163f597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:42:15.849916 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.850660 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b357416-091d-419b-90d4-66fe278262a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:42:15.852381 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1456/2909] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jun 27 15:42:15 2022] DELETE /identity/v3/projects/055f40a719824e82b388537fd163f597 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:42:25.483179 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:25.846775 dv-cinder-847798-1-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:44:35.988379 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:35.990489 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:36.065436 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ae78b-7b15-4a6b-9553-c996680606e0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:36.130604 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26b59442ce5949a59dfe58d1b4b98e6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b85fab3471384942986b8c2cd35c2b9d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:36.131527 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:36.131910 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:36.132233 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:36.135284 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:36.153371 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:36.155477 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfed75-99a6-4d7c-8819-0211dc9558d7 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:36.166373 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1454/2910] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jun 27 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:44:46.161766 dv-cinder-847798-1-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=58 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}